Method and apparatus for data analysis automation testing and optimization, and electronic device

Abstract

This invention provides a method for automated testing and optimization of data analysis, comprising: inputting the data to be tested into a preset machine learning engine for data distribution feature analysis to determine a testing strategy; calling a specific analysis model according to the testing strategy, and initially optimizing the model parameters using the machine learning engine in conjunction with industry characteristics; running the analysis model to generate test results; verifying the test results, and if they do not meet expectations, retesting after iteratively optimizing the model parameters using the machine learning engine, and generating a test report. This invention enables dynamic adjustment of testing strategies to adapt to different data distributions and business scenarios; model parameter optimization, resulting in high model accuracy and stability; and anomaly detection and handling.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a method, apparatus, and electronic device for automated testing and optimization of data analysis. Background Technology

[0002] Automated testing is a process that transforms human-driven testing behaviors into machine-executed actions. For data analysis, existing automated testing tools can typically only verify the correctness of the analysis model, but cannot optimize model parameters or provide improvement suggestions.

[0003] Furthermore, for complex analytical models, such as I-MR control charts, CPK, and two-sample t-tests, it is impossible to dynamically adjust the testing strategy to adapt to different data distributions or business scenarios.

[0004] Therefore, there is an urgent need for an automated testing method that can verify the correctness of the analysis model and effectively optimize the model parameters. Summary of the Invention

[0005] The present invention aims to solve at least one of the above-mentioned technical problems.

[0006] To address the aforementioned problems, this invention provides a method, apparatus, and electronic device for automated testing and optimization of data analysis.

[0007] In a first aspect, the present invention provides a method for automated testing and optimization of data analysis, comprising: The data to be tested is input into a preset machine learning engine to analyze the data distribution characteristics and determine the testing strategy; The specific analysis model is invoked according to the test strategy, and the model parameters are initially optimized by the machine learning engine in combination with industry characteristics. Run the analysis model to generate test results; Verify the test results. If they do not meet expectations, use the machine learning engine to iteratively optimize the model parameters and retest, then generate a test report.

[0008] Optionally, the machine learning engine includes: a clustering algorithm for analyzing data distribution and determining which specific analysis model is suitable for testing; a regression algorithm for optimizing model parameters; and an anomaly detection algorithm for identifying outliers in the test results and providing improvement suggestions.

[0009] Optionally, the judgment test strategy includes: automatically selecting a suitable data transformation method based on data distribution characteristics; and dynamically adjusting the control limits or subgroup size based on process stability.

[0010] Optionally, the verification of the test results, if not meeting expectations, involves retesting after iteratively optimizing the model parameters using the machine learning engine. This includes: when the analysis model is a normal distribution model, the machine learning engine determines whether the data conforms to a normal distribution based on statistics; if not, it performs data transformation or selects another analysis model; when the analysis model is an I-MR control chart model, the machine learning engine determines whether the process is under control based on statistics; if the process is out of control, it adjusts the control limits or checks the data source; when the analysis model is a CPK analysis model, the machine learning engine determines whether the process capability meets the requirements based on the index; if the process capability is insufficient, it suggests adjusting the subgroup size or improving the process.

[0011] Optionally, when the analysis model is a normal analysis model, the generated test report includes the iteratively optimized normal probability plot and statistics; when the analysis model is an I-MR control chart model, the generated test report includes the iteratively optimized control chart and outlier analysis; when the analysis model is a CPK analysis model, the generated test report includes the iteratively optimized process capability index and improvement suggestions.

[0012] Optionally, the criteria for iterative optimization include: a primary objective, which is a hard indicator that makes the data meaningful for analysis; a secondary objective, which is the main optimization direction to make the data conform to the distribution pattern requirements; and a tertiary objective, which can be selectively made as minor adjustments based on the secondary objective.

[0013] Optionally, for the secondary objective, when the analysis model is a normal analysis model, it includes: increasing the concentration of data centers and correcting the two ends of the data distribution; when the analysis model is an I-MR control chart model, it includes: adjusting control limits in different ways according to the degree of autocorrelation and variance stability; when the analysis model is a CPK analysis model, it includes: calculating the value of CPK after multiple random samplings of different subgroups to analyze stability and obtain the optimal subgroup size.

[0014] Optionally, the primary objectives include: data physical validity in accordance with industry experience, data order preservation validity, and engineering interpretability.

[0015] Secondly, the present invention provides an apparatus for automated testing and optimization of data analysis, comprising: The data input module is used to receive test data; An analysis model library with multiple built-in analysis models; The machine learning engine is used to analyze the data distribution characteristics of the tested data, determine the testing strategy, call the specific analysis model, and initially optimize the model parameters in combination with industry characteristics; and verify the test results. If the results do not meet expectations, the model parameters are iteratively optimized. The testing engine is used to run the analysis model and generate test results. The report generation module is used to generate test reports after iterative optimization.

[0016] Thirdly, the present invention provides an electronic device, including a memory and a processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement a method for automated testing and optimization of data analysis as described in the first aspect.

[0017] The technical solution of the present invention can achieve the following by designing a machine learning engine: (1) Dynamically adjusting the test strategy: According to the distribution characteristics of the input data, the machine learning engine automatically selects a suitable test strategy, such as data transformation method, control limit adjustment, etc., so as to adapt to different data distributions and business scenarios; (2) Model parameter optimization: The machine learning engine can automatically optimize the parameters of the analysis model, such as confidence interval, subgroup size, etc., so as to improve the accuracy and stability of the model and reduce human intervention; (3) Anomaly detection and handling: The machine learning engine can identify anomalies in the test results and automatically adjust the model parameters or test strategy to eliminate anomalies. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the automated testing and optimization method for data analysis according to an embodiment of the present invention. Figure 2 This is a system architecture diagram of the data analysis automated testing and optimization apparatus according to an embodiment of the present invention; Figure 3 This is a system architecture diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0020] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0021] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to"; the term "based on" means "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first," "second," etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0024] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for automated testing and optimization of data analysis, comprising: S1. Input the data to be tested into a preset machine learning engine to analyze the data distribution characteristics and determine the testing strategy; S2. Call a specific analysis model according to the test strategy, and perform initial optimization of the model parameters through the machine learning engine; S3. Run the analysis model and generate test results; S4. Verify the test results. If they do not meet expectations, use the machine learning engine to iteratively optimize the model parameters and retest, and generate a test report.

[0025] Specifically, this technical solution will be described in detail in conjunction with actual scenarios.

[0026] Scenario 1: Automated testing and optimization of normal distribution.

[0027] Input a set of raw data, such as CSV or Excel files. The data simulation function generates test data conforming to a specific distribution. The machine learning engine analyzes the data distribution and determines a suitable testing strategy for normality analysis. It then calls the normality analysis model and performs logarithmic transformation optimization on the model parameters. The initial optimization parameter settings can be based on a prior knowledge base. The calculation logic includes: testing methods, significance levels, data preparation (e.g., automatically checking and removing values ​​exceeding physical limits), and baseline targets. The testing engine generates a normal probability plot and statistics. The normal probability plot shows how well the data points fit the theoretical normal distribution, and the statistics include mean, standard deviation, skewness, and kurtosis. Further, the machine learning engine uses the statistics and bias calculations to determine whether the data conforms to a normal distribution. If it does not meet expectations, it performs problem diagnosis and iterative optimization of the model parameters accordingly.

[0028] For example, when the raw data comes from the width of the pin slot in the production process, with a specification of 1.25±0.02mm, the collected data sample is data = [1.254, 1.249, 1.252, 1.247, 1.255, 1.248, 1.253, 1.246, 1.251, 1.250]. Initial observation shows a mean of approximately 1.250, but with significant fluctuations. Based on prior knowledge, some reference angles combining industry requirements or experience are obtained, such as the rightward skewness of the injection molding process, the kurtosis range determined by process characteristics, the physical limits of the mold, and trimming 5% outliers. In this embodiment, the preferred initial optimization calculation logic is: using the Shapiro-Wilk test, with a significance level of α=0.10 and a goodness-of-fit standard of 0.90, automatically removing values ​​less than 1.20 or greater than 1.30, and setting a baseline target.

[0029] A normality analysis model was selected, and the output was validated. This included: (1) P-value bias: the difference between the target P-value threshold of 0.10 and the actual P-value of 0.032 was 0.068, indicating that the actual significance exceeded the requirement; (2) R² bias: the difference between the target R² value of 0.90 and the actual R² value of 0.82 was 0.08, indicating insufficient goodness of fit; (3) Skewness bias: the difference between the actual skewness value of 0.45 and the target upper limit of 0.7 was -0.25, indicating that the skewness was within an acceptable range; (4) Kurtosis bias: the difference between the actual absolute value of kurtosis of 1.8 and the target upper limit of 1.2 was 0.6, indicating that the kurtosis was severely exceeded. In summary, compared with expectations, there were problems of severely insufficient normality, abnormal distribution shape, and poor fitting effect.

[0030] Therefore, the machine learning engine iteratively optimizes based on the three-level objectives: First, regarding the primary objective's rigid constraints, such as physical validity, the rigid limitations determined by mold design and process characteristics dictate that all converted values ​​must be strictly controlled within a physical range of 1.20 to 1.30 millimeters. Regarding data order preservation, the conversion process must maintain the relative order of the original data to ensure data comparability and the accuracy of trend analysis.

[0031] Secondly, the main optimization directions for secondary objectives are as follows: for example, if the current pattern shows a clear low-peak state, the concentration of data centers can be increased by using methods such as Box-Cox transformation with specific parameters, power transformation, and truncation processing; at the same time, the two ends of the data distribution, namely the large value region and the small value region, should be corrected.

[0032] Finally, for secondary optimization directions of the third-level objectives, such as if the current shape is slightly skewed to the left, it can be adjusted by slightly shifting it to the right towards zero.

[0033] Finally, a test report that meets expectations is generated.

[0034] Since this article has elaborated on scenario 1 in detail and uses the same technical solution principle, scenario 2 and scenario 3 will be briefly described.

[0035] Scenario 2: Automated Testing and Optimization of I-MR Control Charts. Input a set of time-series data. The machine learning engine analyzes data fluctuations and dynamically adjusts control limits. For example, an injection molding machine collects temperature data every 5 minutes and finds frequent alarms on the control chart, but the actual process is stable. After analyzing the data, the machine learning engine may determine that the preset rules of the traditional control chart do not match the actual data characteristics of the current process. Therefore, initial control limits are set: Center Line (CL): the average of all data; Upper Control Limit (UCL): mean + 3 standard deviations; Lower Control Limit (LCL): mean - 3 standard deviations.

[0036] The I-MR control chart model is invoked to generate control charts and related statistics. The control charts display data points, control limits, and a center line; the statistics include the mean, range, and standard deviation. The machine learning engine determines whether the process is in control based on the statistics. If the control charts indicate that the process is out of control, the machine learning engine identifies outliers and optimizes and adjusts the control limits, including: The primary objective is to confirm that the data in this sample is time-series data and maintains the correct order and physical validity. For the secondary objective, if significant autocorrelation exists (autocorrelation coefficient > 0.5), the effective standard deviation is recalculated; if the variance is unstable (stability score < 0.7), control limits are set according to the data characteristics of different time periods, such as using sliding windows or piecewise modeling; if there is no significant autocorrelation and the variance is stable, the traditional 3σ control limit is maintained.

[0037] Finally, a test report is generated, including the optimized process capability index and improvement suggestions.

[0038] Scenario 3: Automated Testing and Optimization of CPK Analysis. Input a set of process data, and the machine learning engine analyzes process stability and dynamically adjusts the subgroup size. For example, key dimensional data of the injection molding process (such as part thickness, diameter, etc.), data sample: [1.253, 1.248, 1.251, 1.249, 1.252, 1.247, 1.250, 1.254, ...], unit: millimeters. Initially set 7 possible subgroup sizes: [2, 3, 4, 5, 6, 8, 10].

[0039] The CPK analysis model is invoked to generate process capability indices, such as CP and CPK. The machine learning engine determines whether the process capability meets the requirements based on the indices. If the process capability index does not meet expectations, the machine learning engine optimizes the subgroup size and retests. In this embodiment, the preferred optimization method is to perform multiple random samplings for each selected subgroup size k, extracting enough data to form a complete subgroup from the total data each time, calculating the CPK value based on specification limits, and performing a stability assessment. Ideally, the stability is close to 1, indicating that the CPK values ​​obtained from different samplings are very consistent; in a non-ideal case, the stability is close to 0, indicating that the CPK value fluctuates greatly and the result is unreliable. Through iterative methods, the system quantifies the reliability of CPK calculation for each subgroup size.

[0040] Finally, a test report is generated, including the optimized process capability index and improvement suggestions.

[0041] like Figure 2 As shown, this embodiment of the invention provides a data analysis automated testing and optimization apparatus 300, comprising: Data input module 310 is used to receive test data; Analysis model library 320, with multiple built-in analysis models; The machine learning engine 330 is used to perform data distribution feature analysis on the tested data, determine the testing strategy, call the specific analysis model, and initially optimize the model parameters in combination with industry characteristics; and verify the test results. If the results do not meet expectations, the model parameters are iteratively optimized. Test Engine 340 is used to run the analysis model and generate test results. The report generation module 350 is used to generate test reports after iterative optimization.

[0042] like Figure 3As shown, an electronic device 400 provided in this embodiment of the invention includes a memory 410 and a processor 420; the memory 410 is used to store a computer program; the processor 420 is used to implement a data analysis automated testing and optimization method as described above when the computer program is executed.

[0043] Alternatively, an electronic device 400 includes a memory 410 and a processor 420 coupled to the memory 410; the memory 410 is configured to store a computer program; and the processor 420 is configured to perform the following operations when the computer program is executed: The data to be tested is input into a preset machine learning engine to analyze the data distribution characteristics and determine the testing strategy; The specific analysis model is invoked according to the test strategy, and the model parameters are initially optimized by the machine learning engine in combination with industry characteristics. Run the analysis model and generate test results; Verify the test results. If they do not meet expectations, use the machine learning engine to iteratively optimize the model parameters and retest, then generate a test report.

[0044] The present invention will now be described an electronic device 400 that can serve as a server or client of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. Electronic device 400 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 400 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0045] Electronic device 400 includes a computing unit that can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) or a computer program loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The computing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0046] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. In this application, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units.

[0047] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for automated testing and optimization of data analysis, characterized in that... include: The data to be tested is input into a preset machine learning engine to analyze the data distribution characteristics and determine the testing strategy; The specific analysis model is invoked according to the test strategy, and the model parameters are initially optimized by the machine learning engine in combination with industry characteristics. Run the analysis model to generate test results; Verify the test results. If they do not meet expectations, use the machine learning engine to iteratively optimize the model parameters and retest, then generate a test report.

2. The method for automated testing and optimization of data analysis as described in claim 1, characterized in that... The machine learning engine includes: a clustering algorithm for analyzing data distribution and determining which specific analysis model is suitable for testing; a regression algorithm for optimizing model parameters; and an anomaly detection algorithm for identifying outliers in the test results and providing improvement suggestions.

3. The method for automated testing and optimization of data analysis as described in claim 1, characterized in that... The judgment and testing strategy includes: automatically selecting an appropriate data transformation method based on data distribution characteristics; and dynamically adjusting control limits or subgroup sizes based on process stability.

4. The method for automated testing and optimization of data analysis as described in any one of claims 1-3, characterized in that... The verification of the test results, if not meeting expectations, involves iteratively optimizing the model parameters using the machine learning engine and retesting, including: When the analysis model is a normal distribution model, the machine learning engine determines whether the data conforms to a normal distribution based on statistics. If it does not conform to a normal distribution, the data is transformed or another analysis model is selected. When the analysis model is an I-MR control chart model, the machine learning engine determines whether the process is under control based on statistics. If the process is out of control, the control limits are adjusted or the data source is checked. When the analysis model is a CPK analysis model, the machine learning engine determines whether the process capability meets the requirements based on the index. If the process capability is insufficient, it is recommended to adjust the subgroup size or improve the process.

5. The method for automated testing and optimization of data analysis as described in claim 1, characterized in that... When the analysis model is a normal analysis model, the generated test report includes the iteratively optimized normal probability plot and statistics; when the analysis model is an I-MR control chart model, the generated test report includes the iteratively optimized control chart and outlier analysis. When the analysis model is the CPK analysis model, the generated test report includes the iteratively optimized process capability index and improvement suggestions.

6. The method for automated testing and optimization of data analysis as described in claim 4, characterized in that... The criteria for iterative optimization include: a primary objective, which is a hard indicator that makes the data meaningful for analysis; a secondary objective, which is the main optimization direction to make the data conform to the distribution pattern requirements; and a tertiary objective, which can be selectively made minor adjustments based on the secondary objective.

7. The method for automated testing and optimization of data analysis as described in claim 6, characterized in that... For the secondary objective, when the analysis model is a normal distribution model, it includes: increasing the concentration of data centers and correcting the two ends of the data distribution; when the analysis model is an I-MR control chart model, it includes: adjusting control limits in different ways according to the degree of autocorrelation and variance stability; when the analysis model is a CPK analysis model, it includes: calculating the value of CPK after multiple random samplings of different subgroups to analyze stability and obtain the optimal subgroup size.

8. The method for automated testing and optimization of data analysis as described in claim 6, characterized in that... The primary objectives include: data physical validity in accordance with industry experience, data order preservation validity, and engineering interpretability.

9. A device for automated testing and optimization of data analysis, characterized in that... include: The data input module is used to receive test data; An analysis model library with multiple built-in analysis models; The machine learning engine is used to analyze the data distribution characteristics of the data being tested, determine the testing strategy, call the specific analysis model, and initially optimize the model parameters in combination with industry characteristics. And verify the test results; if they do not meet expectations, iteratively optimize the model parameters. The testing engine is used to run the analysis model and generate test results. The report generation module is used to generate test reports after iterative optimization.

10. An electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement a method for automated testing and optimization of data analysis as described in any one of claims 1 to 8.

Images