One-button big data processing-statistics and result output integrated system

Abstract

The invention relates to the technical field of data statistics and analysis, and discloses a one-button big data processing-statistics and result output integrated system, which comprises a data input module used for receiving and analyzing an original data file uploaded by a user; the data preprocessing module is used for executing automatic processing on the input data, including identification and correction of singular values and missing values in the data and code conversion of classification variables; and the statistical analysis engine is used for automatically selecting and executing a statistical analysis method according to the characteristics of the data, and the statistical analysis method comprises but not limited to hypothesis testing, variance analysis, variance analysis of repeated measurement, logistic regression analysis, LASSO regression analysis and multi-group comparison. The result output module ensures structuralization and meticulous output results, is easy to understand and apply, has more advantages and convenience in big data processing, completes data processing of hundreds of variables, hypothesis testing of statistical description and inter-group comparison and the like within one score, and outputs result files conforming to a paper writing format.

Description

Technical Field

[0001] The present invention relates to the technical field of data statistics and analysis, and in particular to an integrated system for one-click big data processing, statistics and result output. Background Art

[0002] The integrated system of one-click big data processing, statistics and result output is an automated data processing tool widely used in the fields of medicine, scientific research and statistical analysis. Its core function is to help users quickly complete the entire process from data preprocessing to statistical analysis and then to result report generation, simplifying complex statistical operations. Such systems usually support users to upload data files, automatically process them through specified analysis parameters, and generate easy-to-understand analysis reports. This system not only improves the efficiency of data processing, but also reduces the risk of errors caused by manual operations.

[0003] Most of the existing one-click big data processing-statistics and result output integrated systems have basic automated processing capabilities, such as the ability to implement coding conversion of categorical variables, partial interpolation functions in data preprocessing, and simple descriptive statistics and hypothesis testing. Such systems automatically select some statistical analysis methods and generate basic report outputs, reducing the user's analysis workload. At the same time, some systems also support batch processing of data, which can improve the efficiency of data processing to a certain extent. The implementation principle of existing technologies usually relies on an established algorithm library, and the system performs analysis tasks through rule settings and user input.

[0004] However, existing technologies still have shortcomings when faced with complex data preprocessing, statistical method selection and large-scale data processing. First, the data preprocessing function is relatively limited and cannot flexibly cope with various types of missing data processing and outlier correction; second, the selection of statistical analysis methods often lacks automation capabilities, especially when testing conditions such as normality and homogeneity of variance, the existing system fails to accurately identify data characteristics and automatically adjust the analysis method; in addition, the existing system's parallel processing and real-time feedback capabilities are insufficient, and users are prone to long waiting times when processing large-scale data sets, and the results display is not intuitive enough, which limits the efficient use of the system.

[0005] The use of existing technologies requires users to have solid statistical knowledge, and they need to manually select data cleaning and missing value processing methods; they need to manually process the conversion and recoding of categorical variables; they need to manually select methods to judge data characteristics; they need to manually select appropriate hypothesis testing methods based on data characteristics; they need to manually specify groupings, and the analysis of dozens, hundreds, or even thousands of variables is time-consuming and laborious. The reports generated by existing technologies need to be manually sorted. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides an integrated system for one-click big data processing-statistics and result output. The present invention enhances the data preprocessing module to flexibly handle missing values ​​and outliers to ensure data integrity; the statistical analysis engine automatically tests normality and homogeneity of variance, intelligently selects the optimal statistical method, and improves analysis accuracy; supports batch processing and real-time feedback, improves the processing efficiency of large-scale data sets, automatically organizes statistical results, generates comprehensive statistical reports, and outputs the statistical reports as Excel files.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: an integrated system for one-click big data processing, statistics and result output, comprising:

[0008] Data input module, used to receive and parse the original data files uploaded by users;

[0009] Data preprocessing module, which is used to perform automated processing on input data, including identifying and correcting singular values, missing values, and encoding conversion of categorical variables in the data;

[0010] A statistical analysis engine, for automatically selecting and executing statistical analysis methods according to the characteristics of the data, including but not limited to hypothesis testing, analysis of variance, analysis of variance for repeated measures, logistic regression analysis, LASSO analysis, and comparison between multiple groups;

[0011] A composite report generator, used to generate a structured statistical report based on the results of the statistical analysis and support export in multiple formats;

[0012] A user interaction interface, used to receive analysis parameters input by a user;

[0013] The data preprocessing module is further configured to automatically identify data types and perform processing operations according to different characteristics of categorical variables and numerical variables.

[0014] Preferably, the data preprocessing module includes:

[0015] Data verification unit, used to check whether the integrity and format of data meet the analysis requirements;

[0016] Data cleaning unit, used to automatically correct outliers in the data and interpolate missing data using preset rules or machine learning models;

[0017] The code conversion unit is used to convert categorical variables in text form into numerical codes and reclassify the categories according to the threshold set by the user.

[0018] Preferably, the statistical analysis engine comprises:

[0019] A method selection unit, used to automatically select a suitable statistical analysis method according to the statistical characteristics of the input data, including T-test or variance analysis for normally distributed data and non-parametric test for non-normally distributed data;

[0020] The multi-group comparison unit is used to perform multiple comparison corrections and effect size estimation among multiple groups of data, and provide comparative analysis results between and within groups.

[0021] Preferably, the composite report generator comprises:

[0022] Report template unit, used to automatically select predefined report templates, supporting users to customize report content according to their needs;

[0023] The result export unit is used to export the generated report in Excel or other predefined formats, and supports user-defined report saving path.

[0024] Preferably, the composite report generator is further configured to generate a statistical report containing the following contents:

[0025] Descriptive statistics of categorical variables, including the number of samples and percentages in each category;

[0026] Descriptive statistics of numerical variables, including mean, standard deviation, median, quartiles, etc.;

[0027] Hypothesis test results and significance levels for multiple group comparisons.

[0028] Preferably, the user interaction interface includes:

[0029] The parameter input unit is used to receive the analysis parameters specified by the user, such as grouping variables, the range of variables to be analyzed, and data processing options.

[0030] Preferably, the user interaction interface further comprises a wide data input unit for repeated measurements, which is used to support the user to perform paired T-test or variance analysis for repeated measurements.

[0031] Preferably, the system further comprises:

[0032] The missing value processing module is used to process missing data according to the interpolation method selected by the user (such as iterative interpolation or interpolation according to the original numerical ratio) and generate the interpolated data file.

[0033] Preferably, the statistical analysis engine further includes:

[0034] Normality test unit, used to test the normal distribution of numerical variables;

[0035] The variance homogeneity test unit is used to test the variance homogeneity between different groups of data and select appropriate statistical test methods based on the test results.

[0036] The present invention also provides a method for using the integrated system for one-click big data processing, statistics and result output, comprising the following steps:

[0037] Upload the user's data file to the data input module;

[0038] The data is cleaned through the data preprocessing module, including outlier identification, missing value interpolation, and categorical variable encoding;

[0039] The statistical analysis engine performs automated statistical analysis, including normality test, variance homogeneity test, and significance analysis of inter-group differences;

[0040] Generate and output statistical analysis results through a composite report generator, the results including descriptive statistics of numerical variables and categorical variables, hypothesis test results, and multi-group comparison results;

[0041] Supports users to adjust analysis parameters and save analysis results in batches.

[0042] The present invention provides an integrated system for one-click big data processing, statistics and result output.

[0043] Beneficial effects:

[0044] 1. The present invention realizes comprehensive data processing through an automated data preprocessing module, including interpolation of missing values, correction of singular values ​​and automatic encoding of categorical variables, thereby ensuring the integrity and consistency of the data. The system can flexibly process missing data according to the interpolation method selected by the user, making the analysis results more reliable and reducing possible errors caused by human intervention.

[0045] 2. The statistical analysis engine has the automated functions of normality and homogeneity of variance tests, and can intelligently determine the distribution characteristics and variance of the data, and then select the most appropriate statistical analysis method. The system also supports advanced functions such as multi-group comparison and effect size estimation, ensuring the accuracy and scientificity of data analysis, and generating detailed analysis reports.

[0046] 3. The present invention has an intuitive and friendly user interface, supports flexible setting of analysis parameters, and through the batch processing function, the system can process multiple data files at the same time, thereby improving the efficiency of large-scale data set analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a system structure diagram of the present invention;

[0048] Figure 2It is a structural diagram of the data preprocessing module of the present invention;

[0049] Figure 3 is a structural diagram of the statistical analysis engine of the present invention;

[0050] Figure 4 It is a structural diagram of the user interaction interface of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Please see attached Figure 1 -Attached Figure 4 The embodiment of the present invention provides an integrated system for one-click big data processing, statistics and result output, including:

[0053] The data input module is used to receive and analyze the original data files uploaded by users. This module realizes the automatic input and preliminary analysis of data, reduces the workload of users to manually organize data, and improves the usability and convenience of the system;

[0054] Data preprocessing module is used to perform automatic processing on input data, including identification and correction of singular values, missing values ​​and encoding conversion of categorical variables. The data preprocessing module greatly reduces the user's working time on data cleaning, especially when dealing with multi-variable and large data volumes, ensuring data integrity and consistency, and improving the accuracy and reliability of data analysis;

[0055] Statistical analysis engine, which is used to automatically select and execute statistical analysis methods according to the characteristics of the data, including but not limited to hypothesis testing, variance analysis and multi-group comparison. By automatically selecting appropriate statistical methods, the system ensures the scientificity and rigor of the analysis results, reduces the errors caused by users' unfamiliarity with statistical methods, and significantly improves the efficiency of data analysis;

[0056] The composite report generator is used to generate structured statistical reports based on the results of statistical analysis and supports export in multiple formats. The composite report generator realizes the automatic presentation of statistical results, allowing users to quickly obtain professional and formatted reports, reducing the workload of manual data collation and report writing, and greatly improving work efficiency in medical research and clinical research;

[0057] The user interface is used to receive the analysis parameters input by the user, display the analysis progress and output the analysis results. The user interface provides an excellent user experience, allowing non-statistics professionals to easily complete complex statistical analysis tasks, reducing the learning curve and enhancing the practicality and universality of the system.

[0058] Among them, the data preprocessing module is also configured to automatically identify data types and perform processing operations according to the different characteristics of categorical variables and numerical variables. The function of automatically identifying data types further reduces the user's operation complexity and ensures that the system can accurately process different types of data, laying a solid foundation for subsequent statistical analysis.

[0059] Please refer to the attached Figure 1 and attached Figure 2 In a preferred embodiment of the present invention, the data preprocessing module includes:

[0060] The data verification unit is used to check whether the integrity and format of the data meet the analysis requirements. If the verification unit detects that the data is incomplete or the format is incorrect, the system will give a specific error prompt and allow the user to choose to correct or continue processing, thereby effectively ensuring the integrity and consistency of the uploaded data, avoiding errors in subsequent analysis due to format errors or incomplete data, reducing the workload of users to manually check and repair data, and improving the reliability of data analysis;

[0061] The data cleaning unit is used to automatically correct outliers in the data and use preset rules or machine learning models to interpolate missing data. For missing data, the data cleaning unit provides a variety of interpolation methods, including but not limited to mean interpolation, median interpolation, nearest neighbor interpolation, and multiple interpolation. The system will automatically select the optimal interpolation strategy based on the data type and the distribution of missing values, or allow users to manually select the interpolation method. The interpolated data file will be automatically saved and used for subsequent analysis and processing to ensure that data integrity does not affect the results of statistical analysis. The data cleaning unit ensures data consistency and integrity by automatically processing outliers and missing data, reducing the user's time cost on data cleaning and ensuring that subsequent statistical analysis results are more reliable and accurate;

[0062] The encoding conversion unit is used to convert categorical variables in text form into numerical codes, and convert qualified numerical variables into factor variables according to the threshold set by the user. The role of the encoding conversion unit is to automatically convert categorical variables (usually in text form) in the original data file into numerically coded factor variables for subsequent statistical analysis and processing. In addition, the system also provides a threshold customization function. Users can set the threshold of different numbers in a certain numerical variable, so as to convert numerical variables less than or equal to the threshold number into factor variables. For example: in the input data table, there is a variable named "race", which has 9 different values, representing 9 different races. When the threshold we enter is 9 or greater than 9, the system automatically converts the variable "race" into a factor variable, otherwise it remains as the original numerical variable type. All the rules for encoding and category merging will be recorded in the output report to ensure that users can trace back and explain the encoding conversion process, thereby reducing the complexity of manual encoding by users, ensuring that categorical variables can meet the needs of statistical analysis, and improving the accuracy and efficiency of analysis.

[0063] Please refer to the attached Figure 1 and attached Figure 3 In a preferred embodiment of the present invention, the statistical analysis engine includes:

[0064] The method selection unit is used to automatically select a suitable statistical analysis method according to the statistical characteristics of the input data, including T test or variance analysis for normally distributed data, and non-parametric test for non-normally distributed data. For normally distributed data, the method selection unit gives priority to parametric test methods, such as T test (applicable to comparison of two groups of data) or variance analysis (ANOVA, used for comparison of multiple groups of data). These parametric test methods rely on the data conforming to the normal distribution and having homogeneous variance. The satisfaction of these conditions makes the analysis results more accurate and effective. For non-normally distributed data, the method selection unit will automatically switch to non-parametric test methods, such as Mann-Whitney U test (for comparison of two groups of data) or Kruskal-Wallis H test (for comparison of multiple groups of data). The automatic selection mechanism ensures the applicability and scientificity of the statistical method, and greatly improves the accuracy and reliability of the analysis results;

[0065] The multi-group comparison unit is used to perform multiple comparison corrections and effect size estimation between multiple groups of data, and provide inter-group and intra-group comparative analysis results. To prevent the increase of the first type error rate caused by multiple comparisons, the multi-group comparison unit will automatically apply multiple comparison correction methods, such as Bonferroni correction, TukeyHSD (honest significant difference), etc. These methods can adjust the significance level to ensure that a reasonable confidence interval is maintained in multiple comparisons and avoid the emergence of erroneous conclusions. The multi-group comparison unit can not only provide accurate inter-group difference analysis by automating the comparison task of multiple groups of data, but also reduce statistical errors through multiple comparison correction to ensure the reliability and scientificity of the conclusions. The calculation of effect size helps users further understand the practical significance of the results, thereby improving the practicality and explanatory power of data analysis. The multi-group comparison unit also has the ability to analyze intra-group and inter-group data. In intra-group comparison, the system can compare different measurement points (such as different time points) of the same group, which is usually used for the analysis of repeated measurement data. In inter-group comparison, the system compares the data between different groups to determine the differences between the groups. Through these analyses, the system not only provides P values, but also provides detailed statistics, mean differences, effect sizes and other information to help users accurately understand the existence and extent of differences, thereby improving the system's ability to handle complex data analysis, especially for situations with multiple time points, repeated measurements or multiple groups of experimental designs.

[0066] Please refer to the attached Figure 1 In a preferred embodiment of the present invention, the composite report generator comprises:

[0067] The report template unit is used to automatically select predefined report templates and supports users to customize report content according to their needs. The report template unit is the core part of the composite report generator and is responsible for automatically selecting appropriate predefined report templates according to different types of data analysis tasks. When the statistical analysis is completed, the system will automatically identify the type of statistical task currently being performed (such as T test, analysis of variance, non-parametric test, etc.) and match the corresponding report template. For example, if the analysis involves the chi-square test of categorical variables, the system will select the "categorical variable statistical report template" and display the number of samples, frequency, percentage and other information of each group; for descriptive statistics of numerical variables, the system will select the "numerical variable statistical report template" and display common statistics such as mean, standard deviation and median. Through the automated report template selection mechanism, the report template unit simplifies the process of users manually editing and collating reports to ensure that the report content meets the standardized output requirements. In addition, the customization function enhances the flexibility of the system, enabling users to flexibly adjust the report content according to specific needs, thereby improving the applicability and user experience of the report;

[0068] The result export unit is used to export the generated report in Excel or other predefined formats, and supports user-defined report saving paths. The result export unit is responsible for exporting the generated statistical analysis report into common file formats, which is convenient for users to archive, share and further edit. In addition, the result export unit also supports user-defined report saving paths, providing flexibility to meet the storage and processing needs of different users. At the same time, the user-defined save path function improves the convenience and efficiency of file management and reduces the user's time cost in searching and storing reports.

[0069] Please refer to the attached Figure 1 In a preferred embodiment of the present invention, the composite report generator is further configured to generate a statistical report containing the following contents:

[0070] The descriptive statistics of categorical variables include the number of samples and percentages of each category. In the specific implementation process, when the system detects the existence of categorical variables in the data set, it will automatically process these variables and generate a descriptive statistics report for each category. The report will show the number of samples (n) corresponding to each category of each categorical variable and the percentage of samples in this category to the total samples (%). This function automatically generates descriptive statistics for categorical variables, eliminating the need for users to manually calculate or organize categorical data. This intuitive output method greatly simplifies the process of data analysis and report writing, and is particularly suitable for data sets with many categorical variables or large sample sizes.

[0071] Descriptive statistics of numerical variables, including mean, standard deviation, median, quartiles, etc. In the specific implementation, the system first identifies the numerical variables in the input data, and then calculates the descriptive statistics of these variables in turn. For each numerical variable, the system generates a report containing the following information:

[0072] Mean: Displays the average value of the variable, which represents the central tendency of the data.

[0073] Standard deviation: reflects the degree of dispersion of data.

[0074] Median: Displays the middle value of the variable and is used to describe the central position of the data, especially when the data distribution is asymmetric.

[0075] Quartile: includes 25% quantile and 75% quantile, which are used to describe the distribution range and skewness of data. These statistical results are usually presented in tabular form, allowing users to intuitively view the distribution of data, extreme values ​​and outliers, etc.

[0076] By providing comprehensive statistical indicators, it helps users accurately understand the distribution of data and reduces the workload of manual calculation and report writing.

[0077] Hypothesis test results and significance levels of multiple group comparisons. In statistical analysis, hypothesis testing is a key step to determine whether the differences between groups are statistically significant. The composite report generator automatically generates hypothesis testing and significance analysis results, allowing users to quickly identify important differences between groups and ensure the statistical rigor of the results. The significance marking and multiple comparison correction functions reduce the burden of manual adjustment and interpretation of results for users, helping users to understand complex statistical analysis results more intuitively.

[0078] Please see attached Figure 1 and attached Figure 4 In a preferred embodiment of the present invention, the user interaction interface includes:

[0079] The parameter input unit is used to receive the analysis parameters specified by the user, such as grouping variables, the range of variables to be analyzed, and data processing options. The parameter input unit is used to receive the analysis parameters specified by the user. In specific implementation, when the user uploads the data file, the system will automatically parse the column name and variable type of the data. For example, the user can specify the column number of the grouping variable in the "Grouping Variable" column, specify the analysis range in the start column and the end column, specify the number of different values ​​(thresholds) in the numerical variable that will be converted into a categorical variable, specify whether the data needs to be interpolated for missing values, and whether the data needs to be pre-processed. If the input data has been cleaned and has no missing values, enter N in the corresponding position to directly perform the relevant processing of the statistical unit. To ensure that the data processing process meets the research requirements, the parameter input unit provides users with a flexible parameter setting method through a graphical interface, allowing users to easily set statistical analysis parameters according to different analysis requirements, reducing the complexity of users in data selection and configuration, and improving the convenience and flexibility of system operation;

[0080] Please see attached Figure 1 and attached Figure 4 In a preferred embodiment of the present invention, the user interface further includes a batch processing unit for supporting users to upload data files for different statistical purposes. The system can automatically process the data in these files in parallel in the background to complete tasks such as data preprocessing, statistical analysis and report generation. In the specific implementation process, the supported file formats include common formats such as Excel and CSV. The system will automatically perform independent analysis and processing on each file. After uploading the file, the user can set the analysis parameters of each file separately, such as selecting grouping variables, variables to be analyzed and data processing options.

[0081] Please see attached Figure 1 In a preferred embodiment of the present invention, the system further comprises:

[0082] The missing value processing module is used to process missing data according to the interpolation method selected by the user (such as iterative interpolation or interpolation according to the original numerical ratio) and generate the interpolated data file. The missing value processing module is one of the important functional modules in the system, which is mainly used to deal with the missing data problem in the data set. In actual operation, the user can select the appropriate interpolation method on the interface, and the system will automatically apply the interpolation method according to the user's selection and generate a new data file. The processed interpolated data will be generated and saved as a new data file, and the user can use it for subsequent statistical analysis or export and save. Through flexible interpolation method selection, the missing value processing module ensures the integrity of the data, retains the original characteristics of the data as much as possible, and reduces the statistical deviation caused by missing values. The interpolated data file provides a more reliable data foundation for subsequent analysis, ensuring the accuracy and stability of the analysis results;

[0083] The missing value processing module also includes the function of automatically identifying missing values. The system will scan the entire data set during the data preprocessing stage and automatically mark missing values. During the identification process, the system will process them separately according to the type of variable (numerical variable or categorical variable). The missing value identification and automatic interpolation functions simplify the processing of missing data. Users do not need to manually screen and mark missing values. The system can automatically complete these steps and perform interpolation or deletion operations according to the rules set by the user, greatly reducing the complexity and time consumption of data processing. After processing the missing values, the system will automatically generate an interpolation report, which lists the data comparison before and after interpolation in detail, which not only helps users evaluate the effect of the interpolation method, but also enhances the transparency and traceability of the data processing process. Through intuitive comparison and visual reports, users can conduct subsequent analysis more confidently to ensure that the interpolated data is statistically significant.

[0084] Please refer to the attached Figure 1 and attached Figure 3 In a preferred embodiment of the present invention, the statistical analysis engine further comprises:

[0085] The normality test unit is used to test the normal distribution of numerical variables. The normality test unit is one of the important functional modules of the statistical analysis engine and is responsible for testing the normal distribution of numerical variables. Normal distribution is one of the basic assumptions of many statistical methods. For example, T-test and ANOVA (analysis of variance) require that data follow a normal distribution. Therefore, before performing statistical analysis, the system uses the normality test unit to test the normal distribution of each numerical variable to ensure that the appropriate statistical analysis method is selected. The normality test unit automatically detects the normality of the data, ensuring the scientificity and applicability of the subsequent statistical method selection, and avoiding analysis errors caused by data not meeting the assumptions.

[0086] The variance homogeneity test unit is used to detect the variance homogeneity between different groups of data, and select appropriate statistical test methods according to the test results. The variance homogeneity test unit is used to detect whether the variances between different groups of data are consistent, that is, variance homogeneity. Variance homogeneity is one of the necessary conditions for using multiple group comparison tests such as ANOVA (analysis of variance). If the variances between groups are significantly uneven, different test methods (such as Welch's ANOVA or non-parametric tests) need to be used for analysis. Therefore, the variance homogeneity test is a key step to ensure the accuracy of inter-group comparisons. The variance homogeneity test unit automatically detects the variance consistency of different groups of data to ensure that the selected statistical analysis method can adapt to the characteristics of the data, thereby improving the accuracy and reliability of the analysis results. The automatic test and report generation function ensures the transparency and traceability of the normality test and the variance homogeneity test. Users can view the analysis results of each step in detail and quickly judge the distribution characteristics and variance characteristics of the data based on the report. This function reduces the intervention of manual operations, making the statistical analysis process more efficient, while ensuring the scientific accuracy of the analysis results.

[0087] The present invention also provides a method for using the integrated system for one-click big data processing, statistics and result output, which is characterized by comprising the following steps:

[0088] Upload the user's data file to the data input module;

[0089] The data is cleaned through the data preprocessing module, including outlier identification, missing value interpolation, and categorical variable encoding;

[0090] The statistical analysis engine performs automated statistical analysis, including normality test, variance homogeneity test, and significance analysis of inter-group differences;

[0091] Generate and output statistical analysis results through the composite report generator, including descriptive statistics of numerical and categorical variables, hypothesis test results, and multi-group comparison results;

[0092] Supports users to adjust analysis parameters and save analysis results in batches.

[0093] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. One-click big data processing, statistics and result output integrated system, characterized by: include: Data input module, used to receive and parse the original data files uploaded by users; Data preprocessing module, which is used to perform automated processing on input data, including identifying and correcting singular values, missing values, and encoding conversion of categorical variables in the data; A statistical analysis engine, for automatically selecting and executing statistical analysis methods according to the characteristics of the data, including but not limited to hypothesis testing, analysis of variance, and comparison between multiple groups; A composite report generator, used to generate a structured statistical report based on the results of the statistical analysis and support export in multiple formats; A user interaction interface is used to receive analysis parameters input by the user, display analysis progress, and output analysis results; The data preprocessing module is further configured to automatically identify data types and perform processing operations according to different characteristics of categorical variables and numerical variables.

2. The one-click big data processing-statistics and result output integrated system according to claim 1 is characterized in that: The data preprocessing module comprises: Data verification unit, used to check whether the integrity and format of data meet the analysis requirements; Data cleaning unit, used to automatically correct outliers in the data and interpolate missing data using preset rules or machine learning models; The encoding conversion unit is used to convert categorical variables in text form into numerical codes and convert the categories of ordinal numerical variables according to the threshold set by the user.

3. The one-click big data processing-statistics and result output integrated system according to claim 1 is characterized in that: The statistical analysis engine comprises: A method selection unit, used to automatically select a suitable statistical analysis method according to the statistical characteristics of the input data, including T-test or variance analysis for normally distributed data and non-parametric test for non-normally distributed data; The multi-group comparison unit is used to perform multiple comparison corrections and effect size estimation among multiple groups of data, and provide comparative analysis results between and within groups.

4. The one-click big data processing-statistics and result output integrated system according to claim 1 is characterized in that: The composite report generator comprises: Report template unit, used to automatically select predefined report templates, supporting users to customize report content according to their needs; The result export unit is used to export the generated report in Excel or other predefined formats, and supports users to customize the save path of the report.

5. The one-click big data processing-statistics and result output integrated system according to claim 4 is characterized in that: The composite report generator is further configured to generate a statistical report containing the following contents: Descriptive statistics of categorical variables, including the number of samples, percentage, number of samples (percentage), etc. in each category; Descriptive statistics of numerical variables, including mean, standard deviation, median, quartiles, mean ± standard deviation, median (first quartile, third quartile), etc.; Statistics of hypothesis test results, P values, and hypothesis testing methods.

6. The one-click big data processing-statistics and result output integrated system according to claim 1 is characterized in that: The user interaction interface comprises: The parameter input unit is used to receive the analysis parameters specified by the user, such as grouping variables, the range of variables to be analyzed, and data processing options.

7. The one-click big data processing-statistics and result output integrated system according to claim 6, characterized in that: The user interaction interface further includes a repeated measurement data input unit, which is used to support the user to upload a wide data file of repeated measurements and perform paired T test or variance analysis of repeated measurements.

8. The one-click big data processing-statistics and result output integrated system according to claim 1 is characterized in that: The system also includes: The missing value processing module is used to process missing data according to the interpolation method selected by the user (such as iterative interpolation and interpolation based on the existing data ratio) and generate the interpolated data file.

9. The one-click big data processing-statistics and result output integrated system according to claim 1, characterized in that: The statistical analysis engine also includes: Normality test unit, used to test the normal distribution of numerical variables; The variance homogeneity test unit is used to test the variance homogeneity between different groups of data and select appropriate statistical test methods based on the test results.

10. A method for using an integrated system for one-click big data processing, statistics and result output, characterized in that: The integrated system for one-click big data processing, statistics and result output according to any one of claims 1 to 9 comprises the following steps: Upload the user's data file to the data input module; The data is cleaned through the data preprocessing module, including outlier identification, missing value interpolation, and categorical variable encoding; The statistical analysis engine performs automated statistical analysis, including normality test, variance homogeneity test, and significance analysis of inter-group differences; Generate and output statistical analysis results through a composite report generator, the results including descriptive statistics of numerical variables and categorical variables, hypothesis test results, and multi-group comparison results; The analysis parameters can be adjusted through the parameter update button on the user interaction interface, and users are supported to save analysis results in batches.

Images