Performance analysis method and system under KaiwuDB

Abstract

The invention discloses a performance analysis method and system under KaiwuDB, and belongs to the technical field of database intelligent analysis, a gitee pipeline technology is combined with a data insertion and query function of a KaiwuDB database customization function, and an abnormal result analysis and judgment process and an abnormal processing process are added, so that KaiwuDB intelligent performance analysis is realized; comprising the steps of pulling and starting a container mirror image; reading assembly line parameter configuration; pulling code branches and compiling; starting a KaiwuDB database and setting database parameters; running a database insertion scene; operating a database query scene; performing writing and query result analysis; and result exporting and recording. According to the method, the requirement of performance reference stability of a KaiwuDB database write-in and query scene can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of database intelligent analysis, and in particular to a performance analysis method and system under KaiwuDB. Background Art

[0002] Due to the specialized performance customization requirements of the KaiwuDB database, existing traditional manual performance analysis methods cannot fully guarantee KaiwuDB's operational stability, scalability, and performance improvements. Furthermore, due to the limitations of traditional manual performance analysis methods, they are unable to maintain the performance benchmark stability required for KaiwuDB's feature development. This leads to problems such as long processing times, complex workloads, and uncontrollable human errors in operational results. Summary of the Invention

[0003] The technical task of the present invention is to address the above shortcomings and provide a performance analysis method and system under KaiwuDB, which can ensure the performance benchmark stability requirements of KaiwuDB database writing and query scenarios.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] A performance analysis method for KaiwuDB combines Gitee pipeline technology with the data insertion and query capabilities of KaiwuDB database customization functions, and adds abnormal result analysis and judgment, as well as an abnormality handling process, to implement KaiwuDB intelligent performance analysis. The method includes the following steps:

[0006] 1) Pull and start the container image;

[0007] 2) Read pipeline parameter configuration;

[0008] 3) Pull the code branch and compile;

[0009] 4) Start the KaiwuDB database and set the database parameters;

[0010] 5) Run the database insertion scenario;

[0011] 6) Run database query scenarios;

[0012] 7) Write and query result analysis;

[0013] 8) The result is exported and recorded.

[0014] The application of gitee pipeline technology reduces the workload of manual operation and meets the performance benchmark requirements of KaiwuDB database function development scenarios; the combination of gitee pipeline technology and the data insertion and query functions of KaiwuDB database customization functions ensures the performance benchmark stability of KaiwuDB function development; through intelligent result analysis and alarm, it can avoid the complex workload in performance analysis scenarios and uncontrollable human errors in operation results.

[0015] Furthermore, the specific process of implementing KaiwuDB performance analysis by this method is as follows:

[0016] S1. Read the pipeline configuration information and determine whether the pipeline startup parameters are missing; if the parameters are normal, execute step S2; if abnormal, push the relevant abnormality to email or Feishu and other information reminders;

[0017] S2. Image Pull and Start: Pull the specified version of the container image according to the pipeline parameters, and start the container image of the specified hardware specifications according to the configuration parameters;

[0018] S3, code pull and compile: pull the code of the specified branch and compile;

[0019] S4. Start the KaiwuDB database: configure environment variables, start the database node, and configure database parameters;

[0020] S5. Run the database insertion scenario: Automatically generate the insertion script based on the pipeline parameters; insert the performance analysis data into the KaiwuDB database; wait for the writing scenario to complete, calculate the writing speed, and determine whether there are any anomalies; if there are any anomalies, push the insertion anomaly notification to an email reminder or Feishu information reminder; record the writing results to the designated FTP server;

[0021] S6. Run the database query scenario and automatically generate a query script based on the pipeline parameters. Run the query script, wait for the query scenario to complete, calculate the query speed, and determine whether there are any anomalies. If there are any anomalies, send a query anomaly notification to an email or Feishu message reminder; and record the query results to a designated FTP server.

[0022] S7. Customized display performance analysis results of KaiwuDB database.

[0023] Furthermore, the pipeline configuration information includes information such as code version, pipeline function type, running time, database size, result reporting method, etc.

[0024] Furthermore, the pipeline configuration information is read and the startup mode can be selected to run in a scheduled, batch or specified version.

[0025] Furthermore, the insertion script is automatically generated according to the pipeline parameters, and the pipeline parameters include data scale, data type, table structure, insertion method, insertion scenario duration, pressure conditions, etc.

[0026] Furthermore, the query script is automatically generated according to the pipeline parameters, and the pipeline parameters include data size, data type, table structure, query method and query statement type.

[0027] Furthermore, the KaiwuDB database is customized to display performance analysis results, including scheduled task result display, batch pipeline result comparison, abnormal result display, etc.

[0028] The present invention also claims protection for a performance analysis system under KaiwuDB, including a parameter configuration module, a trigger operation module, a result analysis module and a result export module;

[0029] The system implements performance analysis under KaiwuDB through the above method.

[0030] The present invention also claims a performance analysis device under KaiwuDB, comprising: at least one memory and at least one processor;

[0031] The at least one memory is configured to store a machine-readable program;

[0032] The at least one processor is configured to call the machine-readable program to implement the above method.

[0033] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which are capable of implementing the above method when executed by a processor.

[0034] Compared with the prior art, the performance analysis method and system under KaiwuDB of the present invention have the following beneficial effects:

[0035] This method can ensure the performance benchmark stability requirements of KaiwuDB database writing and query scenarios.

[0036] Compared with traditional manual analysis technology, it has three main advantages:

[0037] 1. Meeting the needs of intelligent performance analysis, this solution can reduce the workload of manual performance analysis in the database by about 50%.

[0038] 2. The combination of gitee pipeline technology and the data insertion and query functions of the KaiwuDB database customization function ensures the stability of the KaiwuDB database's performance benchmark.

[0039] 3. Suitable for analyzing performance issues in complex on-site scenarios, it improves the convenience of reproducing problems in complex on-site scenarios. By running multiple pipelines in batches, you can quickly find the functional submission code or functional module that causes performance issues. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is the logical architecture of the KaiwuDB performance pipeline analysis method provided by one embodiment of the present invention;

[0041] Figure 2 This is a flowchart of a performance analysis method under KaiwuDB provided by one embodiment of the present invention;

[0042] Figure 3 This is a diagram of the overall functional logic of KaiwuDB provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be further described below with reference to specific embodiments.

[0044] Although traditional manual performance analysis methods can meet the requirements of stable performance benchmarks as much as possible and can meet the requirements of stable performance benchmarks for most systems, they are unable to meet the special customization and complex performance analysis requirements of the KaiwuDB database, such as intelligent result analysis and early warning; abnormal result alarms; batch specified running of multiple pipeline lines to quickly identify the functional submission code or functional module that causes performance problems, etc. Traditional manual performance analysis methods cannot meet these special customization requirements and cannot provide the requirements for ensuring the stability, scalability, and performance improvement of the KaiwuDB database operation scenarios, which greatly limits the actual needs of the KaiwuDB database such as performance analysis and operation maintenance.

[0045] Based on this, the embodiment of the present invention provides a performance analysis method under KaiwuDB, which combines the gitee pipeline technology with the data insertion and query functions of the KaiwuDB database customization function, and adds abnormal result analysis and judgment, and abnormality handling process to realize KaiwuDB intelligent performance analysis; the key steps are as follows Figure 1 As shown, it includes parameter configuration; triggering operation; result analysis; and result export.

[0046] The application of gitee pipeline technology can reduce the workload of manual operation and meet the performance benchmark requirements of KaiwuDB database function development scenarios; the combination of gitee pipeline technology and the data insertion and query functions of KaiwuDB database customization functions can ensure the performance benchmark stability of KaiwuDB function development; the intelligent result analysis and alarm functions can avoid the complex workload in performance analysis scenarios, uncontrollable human errors in operation results and other problems.

[0047] The implementation of this method includes:

[0048] 1. Pull and start the container image;

[0049] 2. Read the pipeline parameter configuration;

[0050] 3. Pull the code branch and compile;

[0051] 4. Start the KaiwuDB database and set the database parameters;

[0052] 5. Run the database insertion scenario;

[0053] 6. Run database query scenarios;

[0054] 7. Write and query result analysis;

[0055] 8. Export the results and record them.

[0056] Assume that an intelligent performance analysis function module is deployed, such as Figure 2 For simplicity, let's assume that the KaiwuDB database is used for a single-node simple insert and query performance analysis scenario. The KaiwuDB intelligent performance analysis method is as follows:

[0057] S1. Read the pipeline configuration information, including code version, pipeline function type, runtime, database size, and result reporting method. The startup method can be scheduled, batch, or run with a specific version. Determine whether the pipeline startup parameters are missing. If the parameters are normal, proceed to step S2. If not, push the relevant exception to email or Feishu notification.

[0058] S2. Image pull and start: Pull the specified version of the Docker image according to the pipeline parameters, and start the Docker image of the specified hardware specifications according to the configuration parameters.

[0059] S3. Code pull and compile: Pull the code of the specified branch and compile it.

[0060] S4. Start the KaiwuDB database: Configure environment variables, start the database node, and configure database parameters.

[0061] S5. Run the database insert scenario: Automatically generate an insert script based on pipeline parameters such as data size, data type, table structure, insert method, insert scenario duration, and load conditions. Insert performance analysis data into the KaiwuDB database. After the write scenario is completed, calculate the write speed and determine if there are any anomalies. If there are any anomalies, send an insert exception notification to an email or other messaging platform like Lark. Record the write results to a designated FTP server.

[0062] S6. Run database query scenarios and automatically generate query scripts based on pipeline parameters such as data size, data type, table structure, query method, and query statement type. After running the query script and waiting for the query scenario to complete, the system calculates the query speed and determines whether there are any anomalies. If there are any anomalies, a notification is sent to an email or Feishu message. The query results are recorded to a designated FTP server.

[0063] S7. Customized display of performance analysis results for the KaiwuDB database, such as scheduled task results display, batch pipeline result comparison, and abnormal result display.

[0064] In a KaiwuDB database deployment scenario, data reliability, cluster performance, and disaster recovery were compared, simulating a real-world work environment. The results show that this KaiwuDB performance analysis method, based on gitee pipeline technology, meets the functional module requirements for KaiwuDB performance analysis. It ensures the security, scalability, and stability of KaiwuDB query scenarios, maintaining a stable performance benchmark for KaiwuDB query scenarios.

[0065] By combining Gitee pipeline technology with the customized data insertion and query capabilities of the KaiwuDB database, KaiwuDB performance analysis capabilities are provided. This functionality is adaptable to various scenarios (daily performance reporting, abnormal error alarm analysis, abnormal result alarm analysis, etc.), with an accuracy rate exceeding 85%. The application of Gitee pipeline technology improves the convenience of KaiwuDB's performance analysis capabilities, reducing workload by approximately 50% through pipeline timing triggering and intelligent analysis and judgment.

[0066] The present invention also provides a KaiwuDB performance analysis system, including a parameter configuration module, a trigger operation module, a result analysis module, and a result export module. The system implements KaiwuDB performance analysis using the KaiwuDB performance analysis method described in the above embodiment. The specific process is as follows:

[0067] S1. Read the pipeline configuration information, including code version, pipeline function type, runtime, database size, and result reporting method. The startup method can be scheduled, batch, or run with a specific version. Determine whether the pipeline startup parameters are missing. If the parameters are normal, proceed to step S2. If not, push the relevant exception to email or Feishu notification.

[0068] S2. Image pull and start: Pull the specified version of the Docker image according to the pipeline parameters, and start the Docker image of the specified hardware specifications according to the configuration parameters.

[0069] S3. Code pull and compile: Pull the code of the specified branch and compile it.

[0070] S4. Start the KaiwuDB database: Configure environment variables, start the database node, and configure database parameters.

[0071] S5. Run the database insert scenario: Automatically generate an insert script based on pipeline parameters such as data size, data type, table structure, insert method, insert scenario duration, and load conditions. Insert performance analysis data into the KaiwuDB database. After the write scenario is completed, calculate the write speed and determine if there are any anomalies. If there are any anomalies, send an insert exception notification to an email or other messaging platform like Lark. Record the write results to a designated FTP server.

[0072] S6. Run database query scenarios and automatically generate query scripts based on pipeline parameters such as data size, data type, table structure, query method, and query statement type. After running the query script and waiting for the query scenario to complete, the system calculates the query speed and determines whether there are any anomalies. If there are any anomalies, a notification is sent to an email or Feishu message. The query results are recorded to a designated FTP server.

[0073] S7. Customized display of performance analysis results for the KaiwuDB database, such as scheduled task results display, batch pipeline result comparison, and abnormal result display.

[0074] An embodiment of the present invention further provides a performance analysis device under KaiwuDB, comprising: at least one memory and at least one processor;

[0075] The at least one memory is configured to store a machine-readable program;

[0076] The at least one processor is used to call the machine-readable program to implement the performance analysis method under KaiwuDB described in the above embodiment.

[0077] An embodiment of the present invention further provides a computer-readable medium having computer instructions stored thereon. When executed by a processor, the computer instructions implement the performance analysis method under KaiwuDB described in the above embodiment. Specifically, a system or device equipped with a storage medium can be provided, on which software program code implementing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device can be caused to read and execute the program code stored in the storage medium.

[0078] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0079] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0080] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0081] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then based on the instructions of the program code, the CPU installed on the expansion board or expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0082] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.

Claims

1. A performance analysis method under KaiwuDB, characterized in that: Combining Gitee pipeline technology with the data insertion and query capabilities of KaiwuDB's customized database functions, and adding abnormal result analysis and judgment, as well as abnormality handling processes, to implement KaiwuDB intelligent performance analysis. This includes the following steps: 1) Pull and start the container image; 2) Read pipeline parameter configuration; 3) Pull the code branch and compile; 4) Start the KaiwuDB database and set the database parameters; 5) Run the database insertion scenario; 6) Run database query scenarios; 7) Write and query result analysis; 8) The result is exported and recorded.

2. A performance analysis method under KaiwuDB according to claim 1, characterized in that: The specific process of implementing KaiwuDB performance analysis using this method is as follows: S1. Read the pipeline configuration information and determine whether the pipeline startup parameters are missing; if the parameters are normal, execute step S2; if abnormal, push relevant abnormal information reminder; S2. Image Pull and Start: Pull the specified version of the container image according to the pipeline parameters, and start the container image of the specified hardware specifications according to the configuration parameters; S3, code pull and compile: pull the code of the specified branch and compile; S4. Start the KaiwuDB database: configure environment variables, start the database node, and configure database parameters; S5. Run the database insertion scenario: automatically generate the insertion script based on the pipeline parameters; Insert performance analysis data into the KaiwuDB database, wait for the writing scenario to complete, calculate the writing speed, and determine whether there are any anomalies; If there is an exception, push and insert an exception message reminder; Record writing results to the specified FTP server; S6. Run the database query scenario and automatically generate query scripts based on pipeline parameters; Run the query script, wait for the query scenario to complete, calculate the query speed, and determine whether there are any anomalies; If there is an exception, push the query exception message reminder; Record query results to the specified FTP server; S7. Customized display performance analysis results of KaiwuDB database.

3. The performance analysis method under KaiwuDB according to claim 2, characterized in that: The pipeline configuration information includes code version, pipeline function type, running time, database size, and result reporting method information.

4. The performance analysis method under KaiwuDB according to claim 2 or 3, characterized in that: The pipeline configuration information is read and the startup mode can be selected to run in a scheduled, batch or specified version.

5. The performance analysis method under KaiwuDB according to claim 2, characterized in that: The insertion script is automatically generated according to the pipeline parameters, and the pipeline parameters include data scale, data type, table structure, insertion method, insertion scenario duration, and pressure situation.

6. The performance analysis method under KaiwuDB according to claim 2, characterized in that: The query script is automatically generated according to the pipeline parameters, and the pipeline parameters include data scale, data type, table structure, query method and query statement type.

7. The performance analysis method under KaiwuDB according to claim 2, characterized in that: The KaiwuDB database is customized to display performance analysis results, including scheduled task result display, batch pipeline result comparison, and abnormal result display.

8. A performance analysis system under KaiwuDB, characterized in that: Including parameter configuration module, trigger operation module, result analysis module and result export module; The system implements performance analysis under KaiwuDB through the method described in any one of claims 1 to 7.

9. A performance analysis device under KaiwuDB, characterized in that: include: at least one memory and at least one processor; The at least one memory is configured to store a machine-readable program; The at least one processor is configured to call the machine-readable program to implement the method according to any one of claims 1 to 7.

10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, which, when executed by a processor, can implement the method according to any one of claims 1 to 7.

Images