Data processing system based on multi-source data

Abstract

The invention relates to a data processing system based on multi-source data, and the system comprises a user operation interface module which is used for obtaining multi-dimensional data uploaded by a user, obtaining a target analysis task of the multi-dimensional data selected by the user, and obtaining a target machine learning algorithm model selected by the user; the feature screening module is used for selecting features related to the target variable from the multi-dimensional data according to the target variable; the machine learning model module is used for providing machine learning algorithm models corresponding to different analysis tasks for the user; and the model training module is used for training the target machine learning algorithm model according to the related features of the target variable to obtain a target model, so as to perform processing corresponding to the target analysis task on the related features of the target variable based on the target model. By means of the system, a user can be helped to efficiently process and analyze multi-dimensional data, machine learning modeling is automatically carried out, and personalized processing is supported.

Description

Technical Field

[0001] The present invention relates to the field of data analysis and artificial intelligence technology, and in particular to a data processing system based on multi-source data. Background Art

[0002] With the rapid development of big data technology and artificial intelligence (AI), data analysis has been widely used in many industries. In the medical field, with the popularization of electronic medical records, genomic data, imaging data and other health monitoring data, researchers are faced with huge challenges in data processing and analysis. Although traditional statistical methods and data analysis tools have certain effects, they are often difficult to adapt to the dimensions, complexity and diversity of prediction tasks.

[0003] Especially in the medical field, clinical data are often multidimensional and heterogeneous, and often contain missing values ​​and noise. In order to effectively extract useful knowledge from these complex data, researchers need to use machine learning and artificial intelligence technologies to model data, extract features, predict risks, and make personalized recommendations for intervention plans. Therefore, how to build an intelligent analysis platform that integrates data preprocessing, feature screening, machine learning modeling, and predictive evaluation has become a key technical requirement to promote the advancement of medical research and clinical research.

[0004] With the increasing amount of scientific research data in the medical field and the increasing complexity of data structures, data analysis has become a basic and important task in medical research. Medical research data is usually multi-dimensional, highly complex and heterogeneous, involving various types of data such as medical records, genetic data, and imaging data. The analysis and modeling of these data require a high level of technology and professional background. However, the current existing medical data analysis tools generally have the following major shortcomings:

[0005] High technical requirements: Most existing analysis tools require users to have certain programming skills, especially the application of machine learning algorithms, which usually requires users to understand the algorithm's principles, parameter adjustment, data preprocessing and other technical details. For medical researchers, they usually do not have sufficient programming background, which leads to difficulties in using these tools and they cannot fully utilize machine learning and artificial intelligence technologies for data analysis.

[0006] Operational complexity: Although some analysis tools have certain data processing and modeling functions, their operation interfaces are usually complex, the use process is cumbersome, and involves a lot of manual configuration and parameter settings. For users without programming background, this highly complex operation interface greatly increases the threshold for use and affects efficiency.

[0007] Lack of intelligent support: Current tools often lack intelligent functions. Users need to select models, adjust parameters, evaluate results, etc. during use, and there is often no automated optimization and recommendation function. This makes it easy for medical researchers to make inaccurate or unstable results due to improper selection or incomplete analysis when faced with large amounts of data.

[0008] Data processing and analysis processes are separated: Existing tools usually separate processes such as data preprocessing, feature selection, model training, and prediction evaluation. Researchers need to operate them one by one, making the entire analysis process cumbersome and error-prone. This fragmented working mode increases the workload, especially for users who lack data science background. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a data processing system based on multi-source data, aiming to solve at least one of the above technical problems.

[0010] In the first aspect, the technical solution of the present invention to solve the above technical problem is as follows: a data processing system based on multi-source data, the system comprising:

[0011] User interface module, feature screening module, machine learning model module and model training module;

[0012] The user operation interface module is used to obtain the multidimensional data uploaded by the user, obtain the target analysis task of the multidimensional data selected by the user, and obtain the target machine learning algorithm model selected by the user;

[0013] The feature screening module is used to select relevant features of the target variable from the multidimensional data according to the target variable;

[0014] The machine learning model module is used to provide the user with machine learning algorithm models corresponding to different analysis tasks, wherein the different analysis tasks include the target analysis tasks, and the machine learning algorithm models corresponding to the different analysis tasks include the target machine learning algorithm model;

[0015] The model training module is used to train the target machine learning algorithm model according to the relevant characteristics of the target variable to obtain a target model, so as to perform processing corresponding to the target analysis task on the relevant characteristics of the target variable based on the target model.

[0016] The beneficial effects of the present invention are: through the system of the present invention, it is possible to help users efficiently process and analyze multidimensional data, automatically perform machine learning modeling, and support personalized processing. The system can not only reduce the dependence of medical researchers on programming, improve the efficiency and accuracy of data analysis, but also provide more accurate references for clinical decision-making. At the same time, the system integrates multiple analysis modules including user operation interface module, feature screening module, machine learning model module and model training module. Through the one-stop intelligent analysis function, the whole process of data processing can be automatically completed, avoiding the user's switching and complex operations between multiple steps, making the whole analysis process more efficient and coherent.

[0017] Based on the above technical solution, the present invention can also be improved as follows.

[0018] Furthermore, the system also includes a data preprocessing module, which is used to preprocess the multidimensional data, and the preprocessing includes at least one of missing value filling, data standardization, outlier detection and data normalization.

[0019] Furthermore, the user operation interface module is specifically used for:

[0020] Acquire a first trigger operation of a user on a data upload mark displayed on a user operation interface, and acquire the multidimensional data uploaded by the user based on the first trigger operation;

[0021] Acquire a first selection operation of the user on a target analysis task identifier among the analysis task identifiers displayed on the user operation interface, and based on the first selection operation, acquire a target analysis task corresponding to the target analysis task identifier selected by the user;

[0022] Obtain the user's second selection operation on a target machine learning identifier among the machine learning identifiers displayed on the user operation interface, and based on the second selection operation, obtain a target machine learning algorithm model corresponding to the target machine learning identifier selected by the user.

[0023] Furthermore, the system also includes a result display and analysis module, which is used to visually display the processing results output by the model training module.

[0024] Furthermore, the display types of the visual presentation include data distribution graphs, model prediction graphs, and feature importance bar graphs.

[0025] Furthermore, the system also includes a report generation module, which is used to generate a report based on at least one of the output results of the data preprocessing module, the output results of the feature screening module, the output results of the user operation interface module and the output results of the model training module.

[0026] Furthermore, each analysis task corresponds to a processing result, and the user operation interface module is also used to:

[0027] Acquire a second trigger operation of the user for a target processing result identifier among the processing result identifiers displayed on the user operation interface;

[0028] Based on the second trigger operation, the target processing result corresponding to the target processing result identifier is obtained and displayed.

[0029] Furthermore, the user operation interface module is also used for:

[0030] Acquiring the user's setting operation on the target parameter among the various analysis parameters displayed on the user operation interface;

[0031] Based on the setting operation, the set target parameters are obtained and displayed.

[0032] Further, if the number of each of the analysis parameters is greater than the set number, all the analysis parameters are hidden under the menu corresponding to the parameter setting identifier displayed on the user operation interface, and the user operation interface module obtains the user's setting operation for the target parameter in each of the analysis parameters displayed on the user operation interface; based on the setting operation, obtains and displays the set target parameter, specifically for:

[0033] Acquire a third selection operation of the user for the parameter setting identifier, and display a menu corresponding to each of the analysis parameters based on the third selection operation;

[0034] The user's setting operation on the target parameter in each analysis parameter is obtained; based on the setting operation, the set target parameter is obtained and displayed.

[0035] Furthermore, the feature screening module is specifically used to: select relevant features of the target variable from the multidimensional data through a target algorithm according to the target variable, and the target algorithm is any one of a correlation analysis algorithm, a LASSO regression algorithm and a recursive feature elimination algorithm.

[0036] Additional aspects and advantages of the present application will be partially given in the following description, which will become apparent from the following description, or will be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments of the present invention are briefly introduced below.

[0038] Figure 1 A schematic diagram of the structure of a data processing system based on multi-source data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0040] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0041] The embodiment of the present invention provides a possible implementation method, such as Figure 1 As shown, a structural diagram of a data processing system based on multi-source data is provided. Figure 1 As shown in , the system may include:

[0042] User interface module, feature screening module, machine learning model module and model training module;

[0043] The user operation interface module is used to obtain the multidimensional data uploaded by the user, obtain the target analysis task of the multidimensional data selected by the user, and obtain the target machine learning algorithm model selected by the user;

[0044] The feature screening module is used to select relevant features of the target variable from the multidimensional data according to the target variable;

[0045] The machine learning model module is used to provide the user with machine learning algorithm models corresponding to different analysis tasks, wherein the different analysis tasks include the target analysis tasks, and the machine learning algorithm models corresponding to the different analysis tasks include the target machine learning algorithm model;

[0046] The model training module is used to train the target machine learning algorithm model according to the relevant characteristics of the target variable to obtain a target model, so as to perform processing corresponding to the target analysis task on the relevant characteristics of the target variable based on the target model.

[0047] Through the system of the present invention, it is possible to help users efficiently process and analyze multidimensional data, automatically perform machine learning modeling, and support personalized processing. The system can not only reduce the reliance of medical researchers on programming, improve the efficiency and accuracy of data analysis, but also provide a more accurate reference for clinical decision-making. At the same time, the system integrates multiple analysis modules including user interface module, feature screening module, machine learning model module and model training module. Through the one-stop intelligent analysis function, the whole process of data processing can be automatically completed, avoiding the switching and complex operations of users between multiple steps, making the whole analysis process more efficient and coherent.

[0048] The scheme of the present invention is further described below in conjunction with the following specific embodiments. The present invention provides a data processing system based on multi-source data, which aims to solve the above-mentioned shortcomings in the prior art, especially to improve the efficiency and accuracy of medical researchers without programming background in data analysis. Specifically, the present invention solves the defects of the prior art through the following technical means:

[0049] 1) Simplified operation interface and intelligent functions: The user-friendly operation interface provided by the user operation interface module of the present invention enables researchers to quickly perform data analysis and machine learning prediction without programming. The design of the system reduces the operation threshold. Users can complete tasks such as data uploading, preprocessing, model training and evaluation through simple clicks and selection operations, greatly improving the efficiency of data analysis.

[0050] 2) One-stop data analysis and modeling process: The present invention integrates multiple analysis modules such as data preprocessing, feature selection, machine learning modeling, and model evaluation. Through the one-stop intelligent analysis function, the system can automatically complete the entire process of data analysis, avoiding users from switching between multiple steps and complex operations, making the entire analysis process more efficient and coherent.

[0051] 3) Intelligent recommendation and automatic optimization: The system's machine learning model module has multiple machine learning algorithms embedded in it, and has the functions of intelligent model selection and automatic optimization. Through automated model selection and hyperparameter tuning, the system can intelligently recommend appropriate analysis models for different types of tasks (such as classification or regression) and automatically optimize hyperparameters, thus avoiding the tedious work of users manually selecting algorithms and adjusting parameters, reducing human errors in the analysis process, and improving the accuracy of analysis results.

[0052] 4) Machine learning prediction can be achieved without programming: The system of the present invention provides a de-coded operation mode, so that researchers without programming knowledge can easily apply machine learning to perform data analysis. The system automatically completes tasks such as data preprocessing, feature screening, model training and prediction, and researchers can focus on interpreting and making decisions based on analysis results without having to master the details of programming and machine learning algorithms.

[0053] 5) Results display and automatic generation of analysis reports: Through the function of the report generation module, the present invention can automatically generate analysis reports, including data processing, feature screening, model evaluation and prediction results, etc., to help researchers clearly understand the data analysis process and results. The automatic generation of the report not only saves a lot of time, but also avoids the deviation that may occur when manually interpreting the analysis results.

[0054] In summary, the present invention solves the problems of high-threshold technical requirements, operational complexity, insufficient intelligent support, and fragmented data analysis processes in existing medical data analysis tools through a series of technical means. This system enables medical researchers with zero programming knowledge to efficiently perform data preprocessing, modeling, prediction, and result evaluation by simplifying operations, intelligent recommendations, automatic optimization, and de-coding, greatly improving the efficiency and accuracy of data analysis and promoting the development of medical research in a more intelligent and efficient direction.

[0055] Based on the above technical means provided by this system, in this embodiment, a data processing system based on multi-source data provided by this application is described in detail, and the system includes:

[0056] User interface module, feature screening module, machine learning model module and model training module;

[0057] The user operation interface module is used to obtain the multidimensional data uploaded by the user, obtain the target analysis task of the multidimensional data selected by the user, and obtain the target machine learning algorithm model selected by the user;

[0058] The multidimensional data includes medical records, gene data and imaging data; the target analysis task refers to what type of result is obtained after processing the multidimensional data, that is, the type of processing result corresponding to the multidimensional processing. The target machine learning model refers to which machine learning algorithm model is used to process the multidimensional data to obtain the processing result corresponding to the target analysis task.

[0059] As an example, for example, the target analysis task is to extract data that can represent target features from multidimensional data.

[0060] The feature screening module is used to select relevant features of the target variable from the multidimensional data according to the target variable;

[0061] The machine learning model module is used to provide the user with machine learning algorithm models corresponding to different analysis tasks, wherein the different analysis tasks include the target analysis tasks, and the machine learning algorithm models corresponding to the different analysis tasks include the target machine learning algorithm model;

[0062] Among them, the machine learning algorithm models corresponding to different analysis tasks include regression algorithm models (such as linear regression, ridge regression, Lasso regression, etc.), classification algorithm models (such as decision tree, support vector machine, random forest, K nearest neighbor, etc.) and integrated learning method models (such as XGBoost, AdaBoost, etc.), which are used to achieve different analysis tasks.

[0063] The model training module is used to train the target machine learning algorithm model according to the relevant characteristics of the target variable to obtain a target model, so as to perform processing corresponding to the target analysis task on the relevant characteristics of the target variable based on the target model to obtain a processing result.

[0064] Optionally, the main application areas of the system include medical research, clinical data analysis, precision medicine, disease prediction, personalized treatment, etc.

[0065] Optionally, the feature screening module is specifically used to: select relevant features of the target variable from the multidimensional data through a target algorithm according to the target variable, and the target algorithm is any one of a correlation analysis algorithm, a LASSO regression algorithm and a recursive feature elimination algorithm.

[0066] Among them, the target variable represents the parameter corresponding to the processing result corresponding to the target analysis task. For example, if the processing result corresponding to the target analysis task identifies the type of parameter A in the multidimensional data, then the corresponding target variable is parameter A.

[0067] This module automatically selects features related to the target variable based on the characteristics of multidimensional data, and uses statistical methods or machine learning methods (such as correlation analysis, LASSO regression, recursive feature elimination, etc.) to screen features, helping researchers to screen out the most representative features and improve model prediction performance.

[0068] Optionally, the user operation interface module is specifically used to:

[0069] Acquire a first trigger operation of a user on a data upload mark (for example, a button) displayed on a user operation interface, and acquire the multidimensional data uploaded by the user based on the first trigger operation;

[0070] Acquire a first selection operation of the user on a target analysis task identifier among the analysis task identifiers displayed on the user operation interface, and based on the first selection operation, acquire a target analysis task corresponding to the target analysis task identifier selected by the user;

[0071] Obtain the user's second selection operation on a target machine learning identifier among the machine learning identifiers displayed on the user operation interface, and based on the second selection operation, obtain a target machine learning algorithm model corresponding to the target machine learning identifier selected by the user.

[0072] Among them, the target machine learning algorithm model can be selected according to the target analysis task and the relevant characteristics of the target variable.

[0073] The user interface simplifies the complex analysis process and provides users with operable function buttons and options, greatly lowering the usage threshold.

[0074] Optionally, the system further comprises a data preprocessing module, which is used to preprocess the multidimensional data, wherein the preprocessing comprises at least one of missing value filling, data standardization, outlier detection and data normalization.

[0075] This module supports users to preprocess the input multi-dimensional data, including data cleaning processing such as missing value filling, data standardization, and outlier detection, to ensure that the data can meet the input requirements of the machine learning model.

[0076] Optionally, each analysis task corresponds to a processing result, and the user operation interface module is further used to:

[0077] Acquire a second trigger operation of the user for a target processing result identifier among the processing result identifiers displayed on the user operation interface;

[0078] Based on the second trigger operation, the target processing result corresponding to the target processing result identifier is obtained and displayed.

[0079] Optionally, the user operation interface module is further used to:

[0080] Acquiring the user's setting operation on the target parameter among the various analysis parameters displayed on the user operation interface;

[0081] Based on the setting operation, the set target parameters are obtained and displayed.

[0082] Among them, analysis parameters refer to parameters that need to be analyzed. Analysis parameters can be used as target variables. In this application scheme, users are supported to add new analysis parameters and modify the added analysis parameters to provide personalized services for users.

[0083] Furthermore, if the number of each of the analysis parameters is greater than the set number, which means that there is no way to display all the analysis parameters at once on the user operation interface, all the analysis parameters are hidden under the menu corresponding to the parameter setting identifier displayed on the user operation interface, and the user operation interface module obtains the user's setting operation for the target parameter in each of the analysis parameters displayed on the user operation interface; based on the setting operation, obtains and displays the set target parameter, specifically for:

[0084] Acquire a third selection operation of the user for the parameter setting identifier, and display a menu corresponding to each of the analysis parameters based on the third selection operation;

[0085] The user's setting operation on the target parameter in each analysis parameter is obtained; based on the setting operation, the set target parameter is obtained and displayed.

[0086] Optionally, the system further includes a result display and analysis module, which is used to visually display the processing results output by the model training module.

[0087] Furthermore, the display types of the visual presentation include data distribution graphs, model prediction graphs, and feature importance bar graphs.

[0088] The system can visualize the processing results in the form of charts or reports, which is easy for researchers to understand and use. This module supports multi-dimensional result display, including data distribution charts, model prediction charts, feature importance bar charts, etc., to help users intuitively understand the analysis process and results.

[0089] Optionally, the system also includes a report generation module, which is used to generate a report based on at least one of the output results of the data preprocessing module, the output results of the feature screening module, the output results of the user operation interface module and the output results of the model training module.

[0090] Through data analysis and model evaluation results, the system automatically generates a concise and clear analysis report. The report content includes data cleaning steps, feature screening results, model selection, training process, evaluation indicators, etc., which is convenient for scientific researchers to make decisions.

[0091] Optionally, in the model training module, during the model training process, it is also used to evaluate the performance of the model through cross-validation and / or model evaluation (such as accuracy, precision, recall, F1 value, etc.) and provide optimization suggestions to users.

[0092] Optionally, since the data used in the present application is multidimensional data, data of different dimensions have different data characteristics, the feature screening module is specifically used to select relevant features of the target variable from the multidimensional data according to the target variable:

[0093] According to the target variable, determine the relevant features corresponding to the target variable from the medical record data;

[0094] According to the target variable, determine the relevant features corresponding to the target variable from the gene data;

[0095] According to the target variable, a target image corresponding to the target variable is determined from the image data, and the target image is used as a relevant feature corresponding to the target variable.

[0096] Furthermore, according to the target variable, the relevant features corresponding to the target variable can be determined from the medical record data and / or the gene data based on natural language processing technology. Specifically, keywords semantically related to the target variable are determined from the medical record data and / or the gene data, and these semantically related keywords are determined as the relevant features corresponding to the target variable.

[0097] Furthermore, according to the target variable, the target image corresponding to the target variable can be determined from the image data based on image processing technology. Specifically, the target variable can be first converted into a target feature (for example, a feature vector), and then a target area similar to the target feature can be determined from the image data based on the target feature. The target area can be cut out from the image data as a related feature corresponding to the target variable.

[0098] Furthermore, if the data types (text and image) of the relevant features determined from data of different dimensions are different, the relevant features of different data types may be unified into features of the same data type, for example, unified into feature vectors.

[0099] Furthermore, since the multidimensional data includes medical record data, genetic data and imaging data, although the relevant features corresponding to the target variable can be determined from the data of different dimensions, the relevant features determined from the data of different dimensions are not necessarily all related. Based on this, in the present application scheme, after determining the relevant features corresponding to the data of different dimensions, similarity calculation is performed on each relevant feature, and some relevant features with low similarity (less than the set similarity threshold in the similarity calculation result) are excluded based on the similarity calculation results.

[0100] Furthermore, although the present application scheme can perform subsequent processing based on the relevant features corresponding to each multidimensional data, there may be some missing information in the original multidimensional data. In this case, feature expansion processing can be performed based on the relevant features corresponding to each multidimensional data. For example, features similar to the relevant features or target variables can be obtained from other data and added to the relevant features corresponding to each determined multidimensional data.

[0101] Furthermore, the image data may also include different types of images, such as X-ray images, CT images, magnetic resonance imaging, etc.

[0102] For the part corresponding to the target variable, the corresponding parts in different types of images are first aligned, that is, the center point of the part is used as the base point, and the base points in different types of images are overlapped. Then, one image in different types of images is used as the base image, and the other images in different types of images are rotated according to the shape of the part in the base image, so that the shape of the part in the rotated image is the same as the shape of the part in the base image. Finally, based on the target variable, feature extraction can be performed on the rotated image and the base image to obtain the relevant features corresponding to each image.

[0103] Furthermore, the relevant features corresponding to all the images may be fused, and the fused features may be used as the relevant features corresponding to the image data.

[0104] Compared with the prior art, the solution of the present invention has the following beneficial effects:

[0105] (1) Simplified user interface, lowering technical barriers

[0106] Existing technical problems: Existing machine learning analysis tools usually require users to have certain programming knowledge or in-depth understanding of machine learning algorithms. Many researchers (especially medical researchers) do not have a data science background. They can only use statistical software and manually adjust parameters for analysis, which makes data analysis cumbersome and error-prone.

[0107] Advantages of the present invention: The present invention presents complex data analysis and machine learning processes through a graphical interface by designing a concise and intuitive user interface module. Users only need to complete operations such as data upload, task selection, and parameter setting by selecting buttons and drop-down menus, without writing any code. This design enables researchers without programming knowledge to easily perform efficient data analysis.

[0108] Effect analysis: By removing the programming requirement, the threshold for use is greatly lowered, and the user convenience is improved. Especially for non-technical researchers in fields such as medicine, they can focus on the business aspects of data analysis without having to understand the complex algorithm implementation process. This not only speeds up the data analysis process, but also avoids the waste of resources caused by technical barriers.

[0109] (2) Automated data preprocessing to reduce manual intervention

[0110] Existing technical problems: In traditional machine learning analysis tools, data preprocessing usually needs to be performed manually by users, and different data sets require different preprocessing methods. This requires users to have high professional skills in data cleaning, missing value processing, outlier detection and other steps when performing data analysis. This not only increases the difficulty of use, but also easily affects the model effect due to improper operation.

[0111] Advantages of the present invention: The data preprocessing module of the present invention can automatically perform operations such as missing value filling, outlier detection, standardization and normalization of data through an embedded intelligent algorithm. The system will automatically select the appropriate preprocessing method according to the type of input data and task requirements.

[0112] Effect analysis: The module's automated function eliminates the need for researchers to manually intervene in the data cleaning process, reducing the occurrence of human errors and improving data quality. In a large amount of medical data, data preprocessing is a key step that affects the accuracy of analysis. This invention ensures the consistency and standardization of this process, thereby providing more reliable data support for subsequent machine learning model training.

[0113] (3) Intelligent feature screening to optimize model performance

[0114] Existing technical problems: In traditional data analysis, feature selection is usually performed by users based on experience, involving a lot of trial and error. The selection of different features directly affects the predictive ability of the model. Incorrect feature selection may lead to poor model performance, or even overfitting or underfitting.

[0115] Advantages of the present invention: The feature screening module of the present invention automatically evaluates and selects the features most relevant to the target variable through intelligent algorithms (such as correlation analysis, LASSO regression, recursive feature elimination, etc.), helping users to screen out the most representative feature set and maximize the predictive ability of the model.

[0116] Effect analysis: Through automated feature selection, the present invention avoids feature selection errors caused by users’ lack of data processing experience or human interference. This intelligent screening can effectively reduce the impact of redundant features and noise data, and improve the accuracy and stability of the model. Especially when processing complex medical data, the optimization of feature selection can significantly improve the prediction effect of the model and provide a more reliable basis for scientific research.

[0117] (4) Integrate multiple machine learning algorithms and flexibly select models

[0118] Problems with existing technologies: Existing data analysis tools often have fixed algorithm frameworks or provide fewer types of algorithms. If users encounter complex analysis problems, they may not be able to find a suitable algorithm to solve them. Moreover, model selection in existing tools usually requires users to manually select based on experience, lacking intelligent support.

[0119] Advantages of the present invention: The machine learning model module of the present invention has built-in multiple regression, classification and ensemble learning algorithms (such as linear regression, support vector machine, decision tree, random forest, XGBoost, etc.). Users can choose the most suitable algorithm for modeling according to the task type and data characteristics.

[0120] Effect analysis: The flexibility and diversity of the present invention allow users to choose appropriate algorithms for different problems. For complex multi-dimensional data in fields such as medicine, the present invention provides more algorithm options, effectively improving the accuracy and flexibility of analysis.

[0121] (5) Automated report generation to improve the standardization of analysis reports

[0122] Problems with existing technologies: Traditional analysis tools usually output results in the form of charts or tables, and users need to spend a lot of time to organize these results into reports. This is not only time-consuming, but may also lead to omissions or inconsistencies in report content due to human factors.

[0123] Advantages of the present invention: The result display and analysis module of the present invention supports automatic generation of analysis reports, and the report content covers data preprocessing, feature screening, model training and evaluation process, result analysis, etc. The report includes not only numerical indicators, but also visual charts and images, which are convenient for researchers to understand intuitively.

[0124] Effect analysis: Automated report generation avoids omissions, errors, and irregularities that may occur when manually writing reports. Through a systematic report generation process, not only does it improve the standardization and consistency of reports, it also saves researchers a lot of time in report writing, allowing them to focus on higher-level research work. This has played an important role in accelerating the scientific research process and facilitating data sharing.

[0125] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the disclosure scope involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) by each other.

Claims

1. A data processing system based on multi-source data, characterized in that: include: User interface module, feature screening module, machine learning model module and model training module; The user operation interface module is used to obtain the multidimensional data uploaded by the user, obtain the target analysis task of the multidimensional data selected by the user, and obtain the target machine learning algorithm model selected by the user; The feature screening module is used to select relevant features of the target variable from the multidimensional data according to the target variable; The machine learning model module is used to provide the user with machine learning algorithm models corresponding to different analysis tasks, wherein the different analysis tasks include the target analysis tasks, and the machine learning algorithm models corresponding to the different analysis tasks include the target machine learning algorithm model; The model training module is used to train the target machine learning algorithm model according to the relevant characteristics of the target variable to obtain a target model, so as to perform processing corresponding to the target analysis task on the relevant characteristics of the target variable based on the target model.

2. The system according to claim 1, characterized in that The system also includes a data preprocessing module, which is used to preprocess the multidimensional data, and the preprocessing includes at least one of missing value filling, data standardization, outlier detection and data normalization.

3. The system according to claim 1, characterized in that The user operation interface module is specifically used for: Acquire a first trigger operation of a user on a data upload mark displayed on a user operation interface, and acquire the multidimensional data uploaded by the user based on the first trigger operation; Acquire a first selection operation of the user on a target analysis task identifier among the analysis task identifiers displayed on the user operation interface, and based on the first selection operation, acquire a target analysis task corresponding to the target analysis task identifier selected by the user; Obtain the user's second selection operation on a target machine learning identifier among the machine learning identifiers displayed on the user operation interface, and based on the second selection operation, obtain a target machine learning algorithm model corresponding to the target machine learning identifier selected by the user.

4. The system according to claim 1, characterized in that The system also includes a result display and analysis module, which is used to visually display the processing results output by the model training module.

5. The system according to claim 4, characterized in that The display types of the visual presentation include data distribution graphs, model prediction charts, and feature importance bar charts.

6. The system according to any one of claims 1 to 5, characterized in that The system also includes a report generation module, which is used to generate a report based on at least one of the output results of the data preprocessing module, the output results of the feature screening module, the output results of the user operation interface module and the output results of the model training module.

7. The system according to claim 1, characterized in that Each analysis task corresponds to a processing result, and the user operation interface module is also used to: Acquire a second trigger operation of the user for a target processing result identifier among the processing result identifiers displayed on the user operation interface; Based on the second trigger operation, the target processing result corresponding to the target processing result identifier is obtained and displayed.

8. The system according to claim 1, characterized in that The user operation interface module is also used for: Acquiring the user's setting operation on the target parameter among the various analysis parameters displayed on the user operation interface; Based on the setting operation, the set target parameters are obtained and displayed.

9. The system according to claim 8, characterized in that If the number of each of the analysis parameters is greater than the set number, all analysis parameters are hidden under the menu corresponding to the parameter setting identifier displayed on the user operation interface, and the user operation interface module obtains the user's setting operation for the target parameter in each of the analysis parameters displayed on the user operation interface; based on the setting operation, obtains and displays the set target parameter, specifically for: Acquire a third selection operation of the user for the parameter setting identifier, and display a menu corresponding to each of the analysis parameters based on the third selection operation; The user's setting operation on the target parameter in each analysis parameter is obtained; based on the setting operation, the set target parameter is obtained and displayed.

10. The system according to claim 1, characterized in that The feature screening module is specifically used to select relevant features of the target variable from the multidimensional data through a target algorithm according to the target variable, and the target algorithm is any one of a correlation analysis algorithm, a LASSO regression algorithm and a recursive feature elimination algorithm.

Images