Grouping quality management system and method based on dynamic data source

Abstract

The invention relates to the field of group quality management, in particular to a group quality management system and method based on a dynamic data source. Comprising a data source management module, a quality data acquisition module, a quality data storage and management module, a quality data analysis and statistics module, a quality control and decision support module, a user management and authority control module and a system setting and management module which are connected through data interfaces, the data flow, the use frequency and the updating condition are monitored, and an early warning mechanism for data source connection abnormity, data access timeout and data conflict is arranged. According to the method, data integration and sharing can be improved, the accuracy of data quality is improved, user experience is improved, and safety is enhanced. According to the method, quality abnormity and fluctuation can be quickly and accurately found, targeted control measures are taken in time, quality problems are restrained in the bud state, and defective products and waste products are effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of group quality management, and in particular to a group quality management system and method based on dynamic data source. Background Art

[0002] The group quality management system is a comprehensive system used to manage and control quality. It covers the quality-related activities of each department, each link and each subsidiary within the group. By formulating unified quality standards, standardized processes, monitoring mechanisms and evaluation methods, it ensures that the products or services provided by the group can reach the expected quality level and enhance the overall competitiveness and reputation of the group.

[0003] However, the existing group quality management system has inconsistent data formats and standards used by different departments and subsidiaries, making data integration difficult and difficult to form a unified quality view. There may be human errors and irregular data entry in the data collection process, resulting in uneven data quality. The functions and processes are relatively rigid, making it difficult to quickly adapt to changes in the group's business and new quality control requirements. The system interface design is not user-friendly, the operating process is complicated, and data security protection measures are insufficient. There is a risk of data leakage, tampering or cyber attack, and the authority management is not sophisticated enough, threatening the group's business secrets and quality data security. Summary of the invention

[0004] To solve the above problems, the present invention provides a group quality management system and method based on dynamic data source, which can improve data integration and sharing, improve the accuracy of data quality, enhance user experience and strengthen security.

[0005] To achieve the above object, the technical solution adopted by the present invention is: A group quality management system based on dynamic data source, including: data source management module connected by data interface, quality data acquisition module, quality data storage and management module, quality data analysis and statistics module, quality control and decision support module, user management and authority control module and system setting and management module, wherein: The data source management module is used to manage and configure dynamic data sources, and implement dynamic management and switching of multiple data sources, including: Automatically identify the data source type corresponding to the added data according to the needs, determine the identification and basic description of each data source type, configure the detailed configuration and connection parameters of different types of data sources, formulate data source switching rules for business operation scenarios based on rule engine technology, set data source switching rules based on user authority levels and roles, formulate data flow rules between different data sources, monitor the data flow size, usage frequency and update status of different data sources; set up early warning mechanisms for data source connection anomalies, data access timeouts and data conflicts; The quality data acquisition module is used to dynamically select data sources for data entry operations according to different business scenarios and conditions and the switching rules set in the data source management module; The quality data storage and management module is used to classify and store data, regularly back up each quality data and provide recovery function; The quality data analysis and statistics module is used to dynamically extract data from different data sources to complete comprehensive analysis and statistics; The quality control and decision support module is used for quality control plan formulation, quality early warning and exception handling, and decision support; The user management and authority control module is used to manage user information in groups and set different operation authorities for different users; The system settings and management module is used to set system parameters and manage logs.

[0006] Furthermore, for different data sources, the data flow size, usage frequency and update status are monitored, and an early warning mechanism is provided for data source connection abnormalities, data access timeouts and data conflicts, including the following steps: A1: Collect detailed information on potential data sources, set indicators for data quality assessment accuracy, completeness consistency and timeliness, conduct preliminary screening of data sources, conduct stability tests after small-scale trials and tests, and formulate, sign and execute contract content; A2: Configure and deploy Wireshark as a traffic monitoring tool, perform data filtering, compression, and encryption preprocessing on the collected data before storing it, use time series analysis algorithms to analyze the data, set warning thresholds based on the statistical analysis results of historical traffic data, and trigger the warning mechanism when the actual traffic exceeds the warning threshold; A3: Record access logs and perform data statistics. Analyze the statistical access data through the correlation analysis algorithm, set warning thresholds based on system resource conditions, and trigger warnings when the frequency of data source usage exceeds or falls below the warning threshold. A4: When performing data update operations, the update interface provided by the system is called to record update logs, monitor update status, and issue update warnings. In update status monitoring, anomalies in data source updates are detected using a statistical anomaly detection algorithm. A5: According to the heartbeat mechanism between the system and the data source, the data source periodically sends heartbeat packets to the system to prove that the connection is normal. The system also sends a connection test request to the data source to verify the availability of the connection. When the system determines that the data source connection is abnormal, the warning mechanism is triggered. The timeout threshold is set according to system performance and business requirements. The access timeout triggers the warning mechanism. The conflict detection algorithm performs real-time detection on the data. When data conflict is detected, the warning mechanism is triggered. A6: Clean duplicate data, missing values, and erroneous data; verify data format, scope, and logic; back up strategy formulation, tool selection and configuration, and execution and monitoring; perform data recovery operations according to the recovery plan when data is lost or damaged; and conduct recovery drills once a month.

[0007] Furthermore, the method of dynamically selecting a data source for data entry according to the switching rules set in the data source management module based on different business scenarios and conditions includes the following steps: B1: Adjust data format conversion and adaptation according to the data formats of different data sources, apply random forest algorithm to train data filtering rules for real-time data collection, continuously adjust and optimize the filtering rules, strengthen data cleaning during data collection, and automatically identify and handle duplicate data, missing data and abnormal data problems; B2: When collecting data, classify and mark the data according to its source and business type. Based on dynamic data sources, the authority management of data entry and collection is refined according to the roles and responsibilities of different users. In the process of data collection and transmission, encryption technology is used to ensure the security and confidentiality of data. B3: Dynamically allocate data collection tasks according to different data sources and business needs, track the progress of data collection tasks in real time, issue reminders and warnings in a timely manner for collection tasks that are not completed on time or have problems, establish traceability information for each piece of data, record the source of the data, collection time, collection personnel and processing process, record all operations and changes during the data collection process, set up detection tools based on cluster analysis algorithms and anomaly detection algorithms, and use monitoring tools to collect and analyze data quality indicators in real time.

[0008] Furthermore, the data format conversion and adaptation for adjusting the data formats of different data sources include: The input data is parsed through the regular expression matching algorithm, and the interface compatibility test of the newly connected data source and system modules is performed according to the data interface testing and verification mechanism.

[0009] Furthermore, the data collection also includes: The data collection process is optimized by using asynchronous collection and batch processing methods. Based on the key nodes and links in the network, the frequently accessed data is cached on network nodes closer to the user through the deployed content distribution network.

[0010] Furthermore, based on the dynamic data source, the authority management of data entry and collection is refined according to the roles and responsibilities of different users. In the process of data collection and transmission, encryption technology is used to ensure the security and confidentiality of data, including: The data in the transmission process is encrypted by the AES encryption algorithm. The AES algorithm is used for encryption in the data storage process. The encryption key is generated, stored and managed by the key management system during the encryption process. A token-based authentication method is used to establish a user authentication system, and a role-based access control algorithm is used to design a permission management system to monitor user data access behavior in real time and record user operation logs.

[0011] Furthermore, the dynamic extraction of data from different data sources to complete comprehensive analysis and statistical work includes: Use statistical analysis methods to calculate the mean, median, standard deviation, variance, skewness and kurtosis of quality data, perform data mining and predictive analysis through hierarchical clustering algorithms, classify quality data using decision tree algorithms, and establish a relationship model between quality indicators and influencing factors using ridge regression algorithms; present data distribution by drawing frequency distribution histograms of data, construct box plots to display the quartiles and outlier range of data, use correlation analysis to analyze the linear correlation between different quality indicators, and use interactive design to allow users to click, filter and zoom to dynamically explore and analyze visual charts; During data transmission, data is encrypted using a symmetric encryption algorithm. Sensitive data is encrypted when stored. In the system, user requests are checked for permissions through middleware or decorators.

[0012] Furthermore, the data is classified and stored, each quality data is backed up regularly, and a recovery function is provided, including: Deduplication is performed through a hash algorithm, and data from different data sources is integrated and cleaned before storage. The table structure is designed based on data characteristics and access patterns. A distributed database is used for large-scale dynamic data. Full data backup is performed regularly, and incremental backup is performed at preset time intervals.

[0013] Furthermore, the quality control plan formulation, quality warning and exception handling and decision support include: Based on historical quality data and business needs, a quality prediction model is constructed using a neural network algorithm, a quality classification model is constructed using a support vector machine algorithm, and a quality optimization model is constructed using a genetic algorithm. The quality prediction model, quality classification model, and quality optimization model are trained and optimized using the collected data to adjust the model parameters; Collect internal and external quality data, merge and associate data from different sources, integrate data based on common key fields, analyze the integrated data through association rule mining technology, determine the reasonable inspection frequency based on the ATI model, conduct inspections based on systematic sampling, and monitor quality data and process performance data in real time through process capability index; Based on the failure mode and effect analysis method, the potential failure modes, causes and impacts of quality problems are identified, the change indicators of RPN values ​​are tracked to regularly evaluate the effectiveness of risk response measures, and the knowledge base is obtained through historical data of neural network model training to provide suggestions and quality control strategies for decision support. Based on the results, interactive visualization charts are created to display key quality indicators and decision information in real time.

[0014] A group quality management method based on dynamic data source includes the following steps: C1: Configure the data source, define the conditions and rules for switching the data source, use the default data source for data operations when the system cannot match the specific switching rules, and monitor the performance indicators of the connection status, data flow, and response time of each data source in real time; C2: Perform data entry and data verification, classify and store quality data according to different data sources, support retrieval of large amounts of quality data through vectorized indexing technology, and regularly back up quality data in each data source; C3: Extract quality data from multiple data sources, conduct comprehensive analysis and mining through a variety of data analysis tools and algorithms, generate quality statistical reports based on user needs, and support custom report formats; C4: Develop quality control plans based on quality objectives and data analysis results, associate quality control plans with corresponding data sources, set quality warning indicators and thresholds, record and track quality anomalies, support problem troubleshooting, cause analysis, and development and implementation of treatment measures, and provide decision support to management based on quality data analysis and quality control execution; C5: Group users for management, support adding, deleting, modifying and querying users, set different operation permissions for users and user groups based on data source access permissions, data entry permissions, data analysis permissions and report generation permissions, and record the system's operation logs, access logs and error logs.

[0015] The beneficial effects of the present invention are: The present invention can adapt to data of different formats and standards, automatically perform format conversion and standardization processing, and solve integration problems caused by inconsistent data formats. Real-time monitoring of data source status and timely warning, log recording of data in order to better understand the situation and take measures, and reasonably improve and restore problem data. It can automatically identify problem data during the collection process, pre-process by identifying the problem, evaluate and trace data pre-processing by comparing data before and after pre-processing, and use advanced encryption technology and security mechanisms to ensure the security of data during transmission, storage and processing, reduce the risk of data leakage and tampering, protect the group's commercial secrets and quality data security, achieve more sophisticated authority management, strictly control user access to data and operation rights, prevent users from exceeding their authority, and reduce data management risks.

[0016] The present invention can integrate and clean data from different data sources before data storage, avoid storing problematic data, design a suitable table structure to improve the scalability and performance of the system, and ensure the integrity and reliability of the data by regularly backing up the full data and performing incremental backups at preset time intervals. Improve the response speed of the system, quickly process large amounts of data and complex businesses, reduce freezes and freezes, and provide users with a smooth user experience. Through a variety of algorithms, data is analyzed and counted to ensure the accuracy of the results, and intuitive visualization of the analysis results and types makes the system easier to use and operate, reduces the user's learning cost and difficulty of use, and improves work efficiency.

[0017] Through real-time data monitoring and analysis, the present invention can quickly and accurately discover quality anomalies and fluctuations, take targeted control measures in a timely manner, curb quality problems in the bud, effectively reduce the generation of defective and waste products, and reduce quality costs. Rich and accurate data support makes the decision-making basis more sufficient and reliable, helping decision makers to fully understand quality status, market demand, cost-effectiveness and other factors, and make quality decisions that are more in line with corporate strategies and goals. The long-term and comprehensive quality data and analysis results provided by the dynamic data source can help companies gain insight into the trends and laws of quality development, and provide a solid foundation and guidance for the formulation of long-term quality strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of a group quality management system based on dynamic data sources; Figure 2 It is a flow chart of the quality data acquisition module; Figure 3 It is a flowchart of the group quality management method based on dynamic data source. DETAILED DESCRIPTION

[0019] See also Figure 1, Figure 2 and Figure 3 As shown, the present invention relates to a collective quality management system based on a dynamic data source, comprising: a data source management module connected by a data interface, a quality data acquisition module, a quality data storage and management module, a quality data analysis and statistics module, a quality control and decision support module, a user management and authority control module and a system setting and management module, wherein: The data source management module is used to manage and configure dynamic data sources, and implement dynamic management and switching of multiple data sources, including: Automatically identify the data source type corresponding to the added data according to the needs, determine the identification and basic description of each data source type, configure the detailed configuration and connection parameters of different types of data sources, formulate data source switching rules for business operation scenarios based on rule engine technology, set data source switching rules based on user authority levels and roles, formulate data flow rules between different data sources, monitor the data flow size, usage frequency and update status of different data sources; set up early warning mechanisms for data source connection anomalies, data access timeouts and data conflicts; The quality data acquisition module is used to dynamically select data sources for data entry operations according to different business scenarios and conditions and the switching rules set in the data source management module; The quality data storage and management module is used to classify and store data, regularly back up each quality data and provide recovery function; The quality data analysis and statistics module is used to dynamically extract data from different data sources to complete comprehensive analysis and statistics; The quality control and decision support module is used for quality control plan formulation, quality early warning and exception handling, and decision support; The user management and authority control module is used to manage user information in groups and set different operation authorities for different users; The system settings and management module is used to set system parameters and manage logs.

[0020] Specifically, the data interface defines clear data input and output interfaces between modules, and specifies the format, content and transmission method of data. For example, the data collected by the quality data acquisition module is transferred to the quality data storage and management module through the data interface for storage; the quality data analysis and statistics module obtains data from the data storage module through the data interface for analysis and processing.

[0021] In the user management and authority control module, user management supports the functions of adding, deleting, modifying and querying users, manages the basic information of users, and manages users in groups to facilitate the unified setting of authorities and assignment of tasks; in authority control, different operation authorities are set for different users or user groups, such as data source access authorities, data entry authorities, data analysis authorities, report generation authorities, etc., to ensure that users can only operate within their authorized scope and ensure the security and confidentiality of system data. Authority control is based on different authority controls for different users in different modules.

[0022] The system setting and management module, the system parameter setting is to set the basic parameters of the system, such as data storage path, log level, interface display style, etc., and supports online modification and saving of system parameters, which can take effect without restarting the system; it also includes log management to record the system's operation log, access log and error log, which is convenient for system monitoring and maintenance, and provides log query and analysis functions to help administrators quickly locate and solve system problems.

[0023] The data source type corresponding to the added data is automatically identified according to the needs, the identification and basic description of each data source type are clarified, the connection parameters are configured in detail for different types of data sources, data source switching rules are formulated for business operation scenarios based on rule engine technology, data source switching rules are set in combination with user authority levels and roles, and data flow rules between different data sources are formulated. More specifically: Assign a unique identifier to each data source type, such as a digital code or string identifier, and write a detailed basic description for each data source type, including information such as the purpose of the data source, data format, data size, and data update frequency. During the data input or data loading stage, automatically identify the data source type corresponding to the data by parsing the data's format, structure, connection string and other features, and use pattern matching, metadata analysis and other technologies.

[0024] For connection parameters, such as the database host name, port number, database name, user name, and password; for file data sources, determine the file path and file format; for Web API data sources, determine the API endpoint and authentication information, etc. After the user enters the connection parameters, perform parameter validity verification, such as checking whether the database connection can be successfully established, whether the file path exists, and whether the API endpoint is accessible.

[0025] For switching rules, such as switching from a local database to a distributed database when the data volume exceeds a certain threshold, the data source switching rules are configured into the system. They can be managed through configuration files, database tables, or a dedicated rule engine, providing rule editing, adding, deleting, and querying functions so that administrators can flexibly adjust switching rules according to business needs.

[0026] The flow rules are to implement the execution logic of data flow rules in the system to ensure that data flows in the prescribed direction. Based on the established data flow monitoring mechanism, the data flow is monitored in real time to see if it complies with the rules, and abnormal data flow situations are discovered and handled in a timely manner.

[0027] Furthermore, for different data sources, the data flow size, usage frequency and update status are monitored, and an early warning mechanism is provided for data source connection abnormalities, data access timeouts and data conflicts, including the following steps: A1: Collect detailed information on potential data sources, set indicators for data quality assessment accuracy, completeness consistency and timeliness, conduct preliminary screening of data sources, conduct stability tests after small-scale trials and tests, and formulate, sign and execute contract content; A2: Configure and deploy Wireshark as a traffic monitoring tool, perform data filtering, compression, and encryption preprocessing on the collected data before storing it, use time series analysis algorithms to analyze the data, set warning thresholds based on the statistical analysis results of historical traffic data, and trigger the warning mechanism when the actual traffic exceeds the warning threshold; A3: Record access logs and perform data statistics. Analyze the statistical access data through the correlation analysis algorithm, set warning thresholds based on system resource conditions, and trigger warnings when the frequency of data source usage exceeds or falls below the warning threshold. A4: When performing data update operations, the update interface provided by the system is called to record update logs, monitor update status, and issue update warnings. In update status monitoring, anomalies in data source updates are detected using a statistical anomaly detection algorithm. A5: According to the heartbeat mechanism between the system and the data source, the data source periodically sends heartbeat packets to the system to prove that the connection is normal. The system also sends a connection test request to the data source to verify the availability of the connection. When the system determines that the data source connection is abnormal, the warning mechanism is triggered. The timeout threshold is set according to system performance and business requirements. The access timeout triggers the warning mechanism. The conflict detection algorithm performs real-time detection on the data. When data conflict is detected, the warning mechanism is triggered. A6: Clean duplicate data, missing values, and erroneous data; verify data format, scope, and logic; back up strategy formulation, tool selection and configuration, and execution and monitoring; perform data recovery operations according to the recovery plan when data is lost or damaged; and conduct recovery drills once a month.

[0028] Specifically, in step A1, during the preliminary screening of data sources, data sources that obviously do not meet quality requirements are removed, and the screened data sources are tried and tested on a small scale, such as extracting part of the data for verification and analysis, checking the authenticity and reliability of the data, and testing the stability of the data source, including the frequency of data updates, the stability of the connection, the speed of data transmission, etc. When signing contracts and agreements with the selected reliable data source providers, the rights and obligations of both parties are clarified, including data quality standards, update cycles, and data security responsibilities. At the same time, the monitoring requirements for data traffic size, usage frequency, and update status are clarified, as well as the responsibilities and solutions for problems such as data source connection anomalies, data access timeouts, and data conflicts.

[0029] In step A2, the monitoring tool is installed and configured at the connection node between the system and the data source, the network interface to be monitored, the IP address or domain name of the data source to be monitored is set, the frequency of data collection and the granularity of data collection are defined, and the monitoring tool collects data on the size of data flow in real time according to the set frequency and granularity, including the number of data packets, number of bytes, source address, destination address, protocol type and other information, and preprocessing is performed to reduce the overhead of data transmission and storage; the collected data flow information is stored in a relational database or a non-relational database. The stored flow data is analyzed using a time series analysis algorithm to calculate statistical indicators such as the average flow, peak flow, valley flow and standard deviation of each data source, and draw a time series graph, bar chart or pie chart of the flow to intuitively display the changing trend and distribution of the flow. After the early warning mechanism is triggered, the early warning methods include email notification, SMS notification, system pop-up window prompt, etc.

[0030] In step A3, the logging code is embedded in the data source access interface of the system. When there is a data access request, the relevant access information is automatically recorded to ensure the accuracy and completeness of the access log record, and the recorded log data is verified and corrected in real time. The access log is regularly counted to calculate the number of visits, total access duration, average access duration and other indicators for each data source in different time periods. The access data is grouped and counted according to the dimensions of user, functional module, data source, etc., and the use of different data sources by different users and different functional modules is analyzed. The warning threshold can be a fixed value or an adaptive value dynamically calculated based on historical data. The warning notification should provide detailed information on the data source of the warning, the type of warning, the time of the warning, the actual value of the usage frequency and the warning threshold so as to understand the situation and take measures.

[0031] In step A4, the update log recording function is implemented in the system's update interface, and the update information transmitted by the data source is stored in the update log table. The update log records are backed up and archived for subsequent query and audit. The system regularly scans the update log table, checks the update records of each data source, and compares the preset update plan of the data source to determine whether the update of the data source is completed on time or in quantity. Formulate early warning rules for data source updates, such as triggering an early warning when the update delay exceeds a certain time, the amount of updated data is lower than the preset threshold, and continuous update anomalies occur. When the update status of the data source meets the early warning rules, the system triggers the early warning mechanism, and the early warning notification is sent to relevant personnel in a variety of ways. The notification details the data source of the update anomaly, the anomaly type, the time when the anomaly occurred, the expected impact, and other information.

[0032] In step A5, if the heartbeat packet from the data source is not received within the specified time or the connection test request is not responded to, the system determines that the connection is abnormal and immediately triggers the early warning mechanism. When the system initiates a data access request to the data source, the timer starts counting. If the data access response time exceeds the set timeout threshold, the system determines that the data access timeout system triggers the early warning mechanism. The corresponding conflict detection algorithm is matched for data update conflicts, data deletion conflicts, or data insertion conflicts. When data is written or updated, the corresponding conflict detection algorithm is called to perform real-time detection on the data. When a data conflict is detected, the system triggers the early warning mechanism.

[0033] In step A6, for duplicate data processing, hash algorithms or sorting comparison algorithms are used to remove duplicates from the data. For missing values, regression model prediction is used to fill in missing values ​​in the data according to the characteristics of the data and business needs. Data validation rules are established, and logical relationship checks are used to validate the data. For erroneous data that does not meet the validation rules, deletion, correction or marking is used to process it. Regular expressions or specific format validation functions are used to validate the format of the data, and the data values ​​are checked to see if they are within a reasonable range. Data that is out of range is marked or processed. Based on the relationship between business logic and data, the data is validated for logical consistency, and illogical data is corrected or deleted. According to factors such as the importance of the data, update frequency, and data volume, formulate a data backup strategy, including the type of backup, backup cycle, and backup time point; select a tar backup tool and configure it according to the backup strategy; set the backup storage location to ensure the safe storage of the backup data; perform data backup operations regularly in accordance with the formulated backup strategy and configuration; during the backup process, monitor the progress, status, and error information of the backup task in real time to ensure the successful execution of the backup task; formulate a detailed data recovery plan, including the recovery process, steps, required resources, or recovery time target; conduct data recovery tests and drills regularly to verify the feasibility and effectiveness of the recovery plan; during the testing and drills, record problems and deficiencies in the recovery process, optimize and improve the recovery plan; when data loss or damage occurs, perform data recovery operations in accordance with the recovery plan; during the recovery process, closely monitor the progress and status of the recovery task, promptly resolve problems that arise during the recovery process, and ensure that the data can be successfully restored to the specified time point and status.

[0034] Furthermore, the method of dynamically selecting a data source for data entry according to the switching rules set in the data source management module based on different business scenarios and conditions includes the following steps: B1: Adjust data format conversion and adaptation according to the data formats of different data sources, apply random forest algorithm to train data filtering rules for real-time data collection, continuously adjust and optimize the filtering rules, strengthen data cleaning during data collection, and automatically identify and handle duplicate data, missing data and abnormal data problems; B2: When collecting data, classify and mark the data according to its source and business type. Based on dynamic data sources, the authority management of data entry and collection is refined according to the roles and responsibilities of different users. In the process of data collection and transmission, encryption technology is used to ensure the security and confidentiality of data. B3: Dynamically allocate data collection tasks according to different data sources and business needs, track the progress of data collection tasks in real time, issue reminders and warnings in a timely manner for collection tasks that are not completed on time or have problems, establish traceability information for each piece of data, record the source of the data, collection time, collection personnel and processing process, record all operations and changes during the data collection process, set up detection tools based on cluster analysis algorithms and anomaly detection algorithms, and use monitoring tools to collect and analyze data quality indicators in real time.

[0035] Specifically, in step B1, real-time data collection applies random forest algorithm to train data filtering rules, divides the classified and marked data into training set, validation set and test set, uses training set data to train the algorithm so that it learns the characteristics and patterns of the data, automatically generates filtering rules, applies the trained filtering rules to the real-time collected data, regularly conducts manual review of the filtered data, and adjusts and optimizes the filtering rules based on feedback. Duplicate data records are identified by comparing the similarity of the data primary key, unique identification field or data content. For duplicate data, it is processed according to the preset strategy (retain the latest data, retain the earliest data or merge duplicate data) to ensure the uniqueness and accuracy of the data; for missing data, a missing data detection mechanism is established, and fields and records with missing values ​​are identified through data integrity verification rules; for abnormal data, data values ​​beyond the reasonable range are detected through data rationality verification rules and data statistical analysis methods (mean, standard deviation or box plot). An automated script and workflow for data cleaning is established, and the logic and rules of data verification and data cleaning are encapsulated into executable program code to realize the automated processing of the data cleaning process. At the same time, it provides data cleaning monitoring and log recording functions to record the execution process of data cleaning, the amount of data processed, the comparison of data before and after cleaning, the handling of abnormal data, etc., to facilitate the evaluation and tracing of the effectiveness of data cleaning work.

[0036] In step B2, source classification is performed during the data collection process, and a source identification field is added to each piece of data to record the source information of the data. The source identification can be represented in the form of codes, keywords, data source names, etc., and a source classification dictionary table or configuration file is established to associate and map the source identification with the corresponding data source detailed information, so that in the process of data processing and analysis, the source and ownership of the data can be quickly and accurately identified according to the source identification. In the business classification, a business type identification field is added to the quality data to indicate the business type to which the data belongs. A business type classification dictionary table or configuration file is established to associate and map the business type identification with the corresponding business type detailed description, data processing logic, analysis method, etc., so that in the process of data processing and analysis, the data can be accurately classified, processed and analyzed according to the business type identification.

[0037] In step B3, data collection tasks are dynamically assigned according to different data sources and business needs. For example, when a new customer order or purchase contract is signed, the corresponding quality data collection task is automatically generated and assigned to the relevant personnel. The progress of data collection tasks is tracked in real time, and reminders and warnings are issued in time for collection tasks that are not completed on time or have problems, to ensure the smooth progress of data collection. Traceability information is established for each piece of data collected, recording the source of the data, collection time, collection personnel, processing process, etc., so that the source and historical processing process of the data can be quickly traced when quality problems or data disputes arise. All operations and changes in the data collection process are recorded in detail, including data entry, modification, deletion, review and other operations, to form an audit log for data quality monitoring and management, and to meet compliance requirements.

[0038] In combination with the group's quality management goals and business needs, determine the indicators for data quality monitoring, such as data accuracy, completeness, consistency, timeliness and availability. Set specific measurement standards and thresholds for each data quality indicator. For example, for data accuracy, a data validation algorithm can be used to check whether the data complies with the preset rules and formats, and the threshold for data accuracy is set to more than 99%; for data completeness, the proportion of missing values ​​in the data records can be counted, and the threshold for the proportion of missing values ​​can be set to less than 1%; for data consistency, the consistency of the same data in different data sources or systems can be compared, and the threshold for consistency deviation can be set; for data timeliness, the update time of the data can be checked to see if it meets the business requirements, and the threshold for the time interval for data updates can be set; for data availability, the data can be evaluated to see whether it can be correctly read and processed, and the threshold for data availability can be set. Assign corresponding weights to each data quality indicator to reflect its importance in the overall data quality assessment. The allocation of weights can be adjusted according to business needs and the importance of the data.

[0039] The monitoring tool can be integrated into the quality data collection module, or it can interact with the collection module as an independent monitoring system. Using the cluster analysis algorithm, the collected data is grouped according to similarity to quickly find outliers and outliers in the data. Use anomaly detection algorithms (such as statistical anomaly detection algorithms, distance-based anomaly detection algorithms, and density-based anomaly detection algorithms) to conduct in-depth analysis of the data and automatically identify anomalies and errors in the data. When the data quality indicators exceed the set threshold or abnormal data is detected, the monitoring system will issue warnings and prompts in a timely manner, and notify relevant personnel (such as data administrators, quality management personnel, business managers, etc.) through emails, SMS, system pop-ups, etc. for processing. At the same time, record detailed information on data quality issues, including the source, time, type, severity, etc. of the problem data, for subsequent problem tracing and analysis.

[0040] Furthermore, the data format conversion and adaptation for adjusting the data formats of different data sources include: The input data is parsed through the regular expression matching algorithm, and the interface compatibility test of the newly connected data source and system modules is performed according to the data interface testing and verification mechanism.

[0041] Specifically, the interface and method of data entry are expanded to support not only manual input, but also systems or databases from different sources such as departments, customers, and purchasers, to achieve automatic data capture and import. For example, integrate with the business systems of various departments to extract quality data regularly; connect with the customer relationship management system (CRM) to obtain quality data from customer feedback; connect with the purchaser's supply chain management system to obtain quality data of purchased raw materials, etc. Develop data format conversion and adaptation functions for data formats of different data sources to ensure that no matter which source the data comes from, it can be uniformly converted into a data format that can be processed and stored by the quality data acquisition module. Utilize the characteristics of dynamic data sources to establish a real-time data acquisition mechanism, and realize real-time data acquisition and updating for key data sources (such as real-time quality inspection data on the production line, real-time customer feedback data, etc.), to ensure the timeliness and accuracy of the data.

[0042] The data conversion middleware includes a data input interface, a data parsing module, a format conversion module, and a data output interface. For the data input interface, it supports multiple data source connection methods, such as database connection, file reading, and network protocol, to achieve connection with different types of data sources. The input data is parsed through a regular expression matching algorithm to automatically identify the format and structure of the data. For format conversion, according to the pre-set target format requirements, the parsed data in different formats are converted and reorganized to a format that can be processed by the quality data acquisition module. Through the data output interface, the converted unified format data is output to the quality data acquisition module to achieve unified collection and processing of data from different data sources.

[0043] Furthermore, the data collection also includes: The data collection process is optimized by using asynchronous collection and batch processing methods. Based on the key nodes and links in the network, the frequently accessed data is cached on network nodes closer to the user through the deployed content distribution network.

[0044] Specifically, conduct a detailed analysis of the existing data collection process, draw a complete flowchart, and clarify each link and data flow. Find out the bottleneck links in the process, such as the long waiting time for data in network transmission and the complex calculations in the data conversion process. Use asynchronous collection to decompose the data collection task into multiple subtasks so that they can be executed concurrently and reduce the waiting time in the data collection process. For batch data, use batch processing to reduce the number and cost of data processing and improve data collection efficiency.

[0045] Evaluate the current network topology and identify key nodes and links in the network. Increase network bandwidth and upgrade network equipment, such as switches and routers, to ensure the speed and stability of data transmission. Deploy a content distribution network (CDN) to cache frequently accessed data on network nodes closer to users, speed up data distribution and transmission, and reduce data collection delays.

[0046] Furthermore, based on the dynamic data source, the authority management of data entry and collection is refined according to the roles and responsibilities of different users. In the process of data collection and transmission, encryption technology is used to ensure the security and confidentiality of data, including: The data in the transmission process is encrypted by the AES encryption algorithm. The AES algorithm is used for encryption in the data storage process. The encryption key is generated, stored and managed by the key management system during the encryption process. A token-based authentication method is used to establish a user authentication system, and a role-based access control algorithm is used to design a permission management system to monitor user data access behavior in real time and record user operation logs.

[0047] Specifically, a user role system is established to divide users into different roles, such as data entry clerk, data auditor, quality manager, department manager, group senior management, etc., and different data entry and collection permissions are assigned to each role. The role-based access control (RBAC) model is used to manage and control user permissions. A user permission management interface is set up in the system so that administrators can easily assign, modify and revoke permissions for users. At the same time, the inheritance and delegation of permissions are supported. For example, department managers can delegate some permissions to subordinate employees, but the use of the delegated permissions needs to be supervised and managed. Logs are recorded for data entry and collection operations, including user operation time, operation content, operation object, operation results, etc., so as to audit and trace user operation behaviors. At the same time, an abnormal operation alarm mechanism is established. When a user performs an operation beyond the scope of his or her authority, the system can promptly issue an alarm signal and notify the administrator to handle it.

[0048] The collected quality data is encrypted using the symmetric encryption algorithm (AES) to ensure the confidentiality and integrity of the data during storage and transmission. The encryption key is uniformly managed and distributed by the system administrator, and is updated and rotated regularly. During data transmission, the Secure Sockets Layer (SSL) protocol or the Transport Layer Security (TLS) protocol is used to encrypt the data to prevent the data from being stolen or tampered with during network transmission. At the same time, an identity authentication mechanism for data transmission is established to ensure that the identities of the sender and receiver of the data are legal and valid. Physical security protection is performed on data storage devices (such as database servers, data storage arrays, etc.) to prevent the equipment from being stolen, damaged or unauthorized access. At the same time, access to data storage devices is controlled and audited, and information such as the visitor's identity, access time, and access content is recorded. A data backup and recovery mechanism is established to regularly back up quality data and store the backup data in a safe place (such as an off-site data center, tape library, etc.). In the event of data loss, damage or system failure, data can be quickly restored to ensure business continuity and data security.

[0049] Access control and permission management are implemented using a token-based authentication method. When a user logs into the system, the identity authentication system verifies the user's identity information. After the verification is passed, an access token with a certain validity period is generated for the user. The permission management system is designed based on the role-based access control (RBAC) algorithm. According to the group's organizational structure and business needs, different roles (such as administrators, ordinary users, data auditors, etc.) are defined, and different data access rights and operation permissions are assigned to each role. The user's data access behavior is monitored in real time, and the user's operation log is recorded. Through data analysis and anomaly detection algorithms, abnormal access behaviors (such as frequent large-scale data downloads, abnormal access during non-working hours, etc.) are warned and recorded to timely discover potential data security risks.

[0050] Furthermore, the dynamic extraction of data from different data sources to complete comprehensive analysis and statistical work includes: Use statistical analysis methods to calculate the mean, median, standard deviation, variance, skewness and kurtosis of quality data, perform data mining and predictive analysis through hierarchical clustering algorithms, classify quality data using decision tree algorithms, and establish a relationship model between quality indicators and influencing factors using ridge regression algorithms; present data distribution by drawing frequency distribution histograms of data, construct box plots to display the quartiles and outlier range of data, use correlation analysis to analyze the linear correlation between different quality indicators, and use interactive design to allow users to click, filter and zoom to dynamically explore and analyze visual charts; During data transmission, data is encrypted using a symmetric encryption algorithm. Sensitive data is encrypted when stored. In the system, user requests are checked for permissions through middleware or decorators.

[0051] Specifically, statistical analysis methods are used to calculate the mean, median, standard deviation, variance, skewness, kurtosis and other statistical indicators of quality data to understand the central tendency, degree of dispersion and distribution of the data. Hierarchical clustering algorithms are used for data mining and predictive analysis to group quality data according to similarity and discover different quality patterns and categories; decision tree algorithms are used to classify quality data to determine whether the product quality is qualified or not or the quality grade; ridge regression algorithms are used to establish a relationship model between quality indicators and influencing factors to conduct quality prediction and trend analysis.

[0052] Generate visual charts based on analysis results and types, allowing users to select operations for data exploration and interaction. Use CSS to build a user interaction interface to integrate the generated visual charts into a unified web interface, and provide data filtering, sorting, and drilling functions. Design a variety of visual charts based on analysis needs, such as line charts to show the changing trend of quality indicators over time, bar charts to compare the quality levels of different products or production lines, pie charts to show the category distribution of quality problems, and box and whisker charts to analyze the discrete degree of quality data. Use interactive design to allow users to dynamically explore and analyze visual charts through operations such as clicking, filtering, and zooming. For example, add mouse hovering to display data point details on line charts, and support sorting and group switching on bar charts.

[0053] During data transmission and storage, encryption algorithms are used to encrypt quality data to ensure data confidentiality and integrity. A user authentication and authorization system is established to assign different data access and operation permissions according to user roles (such as administrators, data analysts, and ordinary users). For example, administrators have the highest permissions and can perform data addition, deletion, modification, and query, algorithm configuration, and system management; data analysts can perform data analysis and visualization operations, but cannot modify the original data; and ordinary users can only view authorized data and analysis results.

[0054] Perform permission checks on user requests through middleware or decorators: When users send requests to the system, middleware will intercept these requests and check the identity and permission information of the user making the request, such as the user's role, the department the user belongs to, and the specific operation permissions granted to the user. When a user initiates a request to trigger a related function or method, the decorator function will first perform a permission check on the user's request before executing the original function. If the user has the corresponding permission, the original function will be executed; if the user does not have permission, the execution can be interrupted or an error prompt will be returned.

[0055] Furthermore, the data is classified and stored, each quality data is backed up regularly, and a recovery function is provided, including: Deduplication is performed through a hash algorithm, and data from different data sources is integrated and cleaned before storage. The table structure is designed based on data characteristics and access patterns. A distributed database is used for large-scale dynamic data. Full data backup is performed regularly, and incremental backup is performed at preset time intervals.

[0056] Specifically, through a specific hash function, the data is mapped to a hash value of a fixed length. Data with the same content will produce the same hash value. The SHA-1 hash function is selected. For the data set to be deduplicated, the hash value of each data element is calculated through the hash function and stored in a hash table. When processing a new data element, its hash value is calculated first, and then the hash table is checked to see if the same hash value already exists. If so, it means that the data element is duplicated and is removed; if not, the hash value is added to the hash table and the data element is retained.

[0057] First, identify the different data sources that need to be integrated, understand the data format, field definition, and data content of each data source, establish data mapping relationships between different data sources based on the semantics of the data and business needs, determine how to associate and integrate data from different sources, and use database connection operations, data integration tools, or programming to merge and connect data. For missing value processing in data cleaning, if the number of missing values ​​is small and has little impact on data analysis, directly delete the records containing missing values, and use the mean, median, mode, or predicted values ​​based on other data features to fill in the missing values. For outlier processing, use clustering algorithms to divide data into different clusters, and data points that fall into small clusters or are far away from the main cluster can be regarded as outliers.

[0058] According to the characteristics and access patterns of the data, design a suitable table structure, including field type, length, constraints, etc. For example, for fields that are frequently queried and connected, set indexes reasonably to improve query efficiency. For large-scale dynamic data, use a distributed database to store and manage data to improve the scalability and performance of the system.

[0059] A full data backup is a complete backup of the entire database or data set, including all data and database objects. According to the importance and frequency of changes of the data, select an appropriate full backup time interval, such as once a month. Before the backup begins, temporarily stop writing to the database to ensure the consistency of the backup data. After the backup is completed, verify the backup data to check the integrity and availability of the backup data. Incremental backup only backs up data that has changed since the last backup. After each full or incremental backup is completed, the backup time point and version information are recorded, and the insertion, update and deletion operations of the data are monitored through the database log files, triggers or other mechanisms. Recovery drills can also be performed as needed.

[0060] Furthermore, the quality control plan formulation, quality warning and exception handling and decision support include: Based on historical quality data and business needs, a quality prediction model is constructed using a neural network algorithm, a quality classification model is constructed using a support vector machine algorithm, and a quality optimization model is constructed using a genetic algorithm. The quality prediction model, quality classification model, and quality optimization model are trained and optimized using the collected data to adjust the model parameters; Collect internal and external quality data, merge and associate data from different sources, integrate data based on common key fields, analyze the integrated data through association rule mining technology, determine the reasonable inspection frequency based on the ATI model, conduct inspections based on systematic sampling, and monitor quality data and process performance data in real time through process capability index; Based on the failure mode and effect analysis method, the potential failure modes, causes and impacts of quality problems are identified, the change indicators of RPN values ​​are tracked to regularly evaluate the effectiveness of risk response measures, and the knowledge base is obtained through historical data of neural network model training to provide suggestions and quality control strategies for decision support. Based on the results, interactive visualization charts are created to display key quality indicators and decision information in real time.

[0061] Specifically, use the keras library to build a multi-layer perceptron (MLP) neural network, define the network structure, including the input layer, hidden layer (number of neurons, activation function) and output layer, select the optimizer (such as Adam optimizer) and loss function (such as mean square error), use training data to train the model, and update the weights and biases of the network through the back propagation algorithm. Use the SVC class of the scikit-learn library to build a support vector machine model, select the kernel function (polynomial kernel), adjust the parameters of the kernel function and the penalty parameter C, use the training data to train the SVM model, and find the optimal hyperplane to separate different categories of data. Define the encoding method of the problem, encode the quality control parameters into chromosomes, determine the fitness function, calculate the fitness value of the chromosome according to the quality objectives (such as cost minimization, quality maximization), set the parameters of the genetic algorithm (population size, crossover probability, mutation probability, number of iterations, etc.), use selection, crossover and mutation operations to evolve the population, and find the optimal combination of quality control parameters.

[0062] Collect internal quality data (such as production process data, test data, customer feedback data) and external data (such as market research data, industry standard data, competitor data), use the pandas library to merge and associate data from different sources, and integrate data based on common key fields (such as product model, production batch, timestamp, etc.). Use the apyori library to mine association rules and discover the association between different quality indicators. Set the minimum support and minimum confidence thresholds to mine strong association rules that meet the conditions, such as discovering the association between a certain production process parameter and the product defect rate. Cluster analysis: Use the DBSCAN clustering algorithm of the scikit-learn library to cluster quality data, determine the number of clusters or select the optimal clustering result based on the clustering effectiveness index, and classify similar quality data into one category to discover different quality patterns and groups. Use the PCA class of the scikit-learn library to perform principal component analysis to reduce the dimensionality of high-dimensional quality data. Extract the main components and reduce the dimensionality of the data while retaining the main features of the data for easy data visualization and analysis. According to the product design requirements, customer needs and industry standards, determine the key indicators and standard limits of quality control, and use statistical process control methods to formulate quality control standards; according to the product quality stability and production batch, use the ATI model to determine the reasonable inspection frequency, and weigh the importance of the product and the inspection cost to choose sampling inspection or full inspection; monitor quality data and process performance data in real time through the process capability index. When the process changes or the quality indicators exceed the control limit, use the feedback control mechanism to dynamically adjust the quality control strategy, such as tightening the inspection standard, increasing the inspection frequency, adjusting the control limit, etc.

[0063] Based on the failure mode and effect analysis method, the potential failure modes, causes and effects of quality problems are identified, and each failure mode is assigned a score of severity (S), occurrence (O) and detection (D). The risk priority number (RPN = S × O × D) is used to assess the priority of risks. For high-risk quality problems, preventive measures (such as improving design, strengthening personnel training, and optimizing processes) and emergency measures (such as product recalls, quality rectification, and customer compensation) are formulated, risk monitoring indicators are established, and the effectiveness of risk response measures is regularly evaluated, such as tracking changes in RPN values ​​and checking the implementation of preventive measures.

[0064] Use the flask framework to build a web application and design a user-friendly decision support interface. Display data analysis results, decision model recommendations, quality control strategies and other information on the interface, and provide data filtering, sorting, query and other functions. Use the matplotlib library to draw line charts, bar charts, box plots, scatter plots, etc. to show the distribution, trends and relationships of quality data. Use the plotly library to create interactive visualization charts, such as dynamic quality control charts and scalable data analysis charts, to support users to explore data through mouse operations. Use the dash library to build a web-based data analysis dashboard, integrate multiple visualization charts on one page, and display key quality indicators and decision information in real time.

[0065] A group quality management method based on dynamic data source includes the following steps: C1: Configure the data source, define the conditions and rules for switching the data source, use the default data source for data operations when the system cannot match the specific switching rules, and monitor the performance indicators of the connection status, data flow, and response time of each data source in real time; C2: Perform data entry and data verification, classify and store quality data according to different data sources, support retrieval of large amounts of quality data through vectorized indexing technology, and regularly back up quality data in each data source; C3: Extract quality data from multiple data sources, conduct comprehensive analysis and mining through a variety of data analysis tools and algorithms, generate quality statistical reports based on user needs, and support custom report formats; C4: Develop quality control plans based on quality objectives and data analysis results, associate quality control plans with corresponding data sources, set quality warning indicators and thresholds, record and track quality anomalies, support problem troubleshooting, cause analysis, and development and implementation of treatment measures, and provide decision support to management based on quality data analysis and quality control execution; C5: Group users for management, support adding, deleting, modifying and querying users, set different operation permissions for users and user groups based on data source access permissions, data entry permissions, data analysis permissions and report generation permissions, and record the system's operation logs, access logs and error logs.

[0066] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A group quality management system based on dynamic data source, characterized in that: include: The data interface connects the data source management module, quality data acquisition module, quality data storage and management module, quality data analysis and statistics module, quality control and decision support module, user management and authority control module and system setting and management module, among which: The data source management module is used to manage and configure dynamic data sources, and implement dynamic management and switching of multiple data sources, including: Automatically identify the data source type corresponding to the added data according to the needs, determine the identification and basic description of each data source type, configure the detailed configuration and connection parameters of different types of data sources, formulate data source switching rules for business operation scenarios based on rule engine technology, set data source switching rules based on user authority levels and roles, formulate data flow rules between different data sources, monitor the data flow size, usage frequency and update status of different data sources; set up early warning mechanisms for data source connection anomalies, data access timeouts and data conflicts; The quality data acquisition module is used to dynamically select data sources for data entry operations according to different business scenarios and conditions and the switching rules set in the data source management module; The quality data storage and management module is used to classify and store data, regularly back up each quality data and provide recovery function; The quality data analysis and statistics module is used to dynamically extract data from different data sources to complete comprehensive analysis and statistics; The quality control and decision support module is used for quality control plan formulation, quality early warning and exception handling, and decision support; The user management and authority control module is used to manage user information in groups and set different operation authorities for different users; The system settings and management module is used to set system parameters and manage logs.

2. The group quality management system based on dynamic data source according to claim 1 is characterized in that: The data flow size, usage frequency and update status of different data sources are monitored, and an early warning mechanism is provided for data source connection abnormalities, data access timeouts and data conflicts, including the following steps: A1: Collect detailed information on potential data sources, set indicators for data quality assessment accuracy, completeness consistency and timeliness, conduct preliminary screening of data sources, conduct stability tests after small-scale trials and tests, and formulate, sign and execute contract content; A2: Configure and deploy Wireshark as a traffic monitoring tool, perform data filtering, compression, and encryption preprocessing on the collected data before storing it, use time series analysis algorithms to analyze the data, set warning thresholds based on the statistical analysis results of historical traffic data, and trigger the warning mechanism when the actual traffic exceeds the warning threshold; A3: Record access logs and perform data statistics. Analyze the statistical access data through the correlation analysis algorithm, set warning thresholds based on system resource conditions, and trigger warnings when the frequency of data source usage exceeds or falls below the warning threshold. A4: When performing data update operations, the update interface provided by the system is called to record update logs, monitor update status, and issue update warnings. In update status monitoring, anomalies in data source updates are detected using a statistical anomaly detection algorithm. A5: According to the heartbeat mechanism between the system and the data source, the data source periodically sends heartbeat packets to the system to prove that the connection is normal. The system also sends a connection test request to the data source to verify the availability of the connection. When the system determines that the data source connection is abnormal, the warning mechanism is triggered. The timeout threshold is set according to system performance and business requirements. The access timeout triggers the warning mechanism. The conflict detection algorithm performs real-time detection on the data. When data conflict is detected, the warning mechanism is triggered. A6: Clean duplicate data, missing values, and erroneous data; verify data format, scope, and logic; back up strategy formulation, tool selection and configuration, and execution and monitoring; perform data recovery operations according to the recovery plan when data is lost or damaged; and conduct recovery drills once a month.

3. The group quality management system based on dynamic data source according to claim 1 is characterized in that: The method of dynamically selecting a data source for data entry according to different business scenarios and conditions and the switching rules set in the data source management module includes the following steps: B1: Adjust data format conversion and adaptation according to the data formats of different data sources, apply random forest algorithm to train data filtering rules for real-time data collection, continuously adjust and optimize the filtering rules, strengthen data cleaning during data collection, and automatically identify and handle duplicate data, missing data and abnormal data problems; B2: When collecting data, classify and mark the data according to its source and business type. Based on dynamic data sources, the authority management of data entry and collection is refined according to the roles and responsibilities of different users. In the process of data collection and transmission, encryption technology is used to ensure the security and confidentiality of data. B3: Dynamically allocate data collection tasks according to different data sources and business needs, track the progress of data collection tasks in real time, issue reminders and warnings in a timely manner for collection tasks that are not completed on time or have problems, establish traceability information for each piece of data, record the source of the data, collection time, collection personnel and processing process, record all operations and changes during the data collection process, set up detection tools based on cluster analysis algorithms and anomaly detection algorithms, and use monitoring tools to collect and analyze data quality indicators in real time.

4. The group quality management system based on dynamic data source according to claim 3 is characterized in that: The data format conversion and adaptation according to the data formats of different data sources include: The input data is parsed through the regular expression matching algorithm, and the interface compatibility test of the newly connected data source and system modules is performed according to the data interface testing and verification mechanism.

5. The group quality management system based on dynamic data source according to claim 3 is characterized in that: The data collection also includes: The data collection process is optimized by using asynchronous collection and batch processing methods. Based on the key nodes and links in the network, the frequently accessed data is cached on network nodes closer to the user through the deployed content distribution network.

6. The group quality management system based on dynamic data source according to claim 3 is characterized in that: Based on the dynamic data source, the authority management of data entry and collection is refined according to the roles and responsibilities of different users. In the process of data collection and transmission, encryption technology is used to ensure the security and confidentiality of data, including: The data in the transmission process is encrypted by the AES encryption algorithm. The AES algorithm is used for encryption in the data storage process. The encryption key is generated, stored and managed by the key management system during the encryption process. A token-based authentication method is used to establish a user authentication system, and a role-based access control algorithm is used to design a permission management system to monitor user data access behavior in real time and record user operation logs.

7. The group quality management system based on dynamic data source according to claim 1 is characterized in that: The dynamic extraction of data from different data sources to complete comprehensive analysis and statistical work includes: Use statistical analysis methods to calculate the mean, median, standard deviation, variance, skewness and kurtosis of quality data, perform data mining and predictive analysis through hierarchical clustering algorithms, classify quality data using decision tree algorithms, and establish a relationship model between quality indicators and influencing factors using ridge regression algorithms; present data distribution by drawing frequency distribution histograms of data, construct box plots to display the quartiles and outlier range of data, use correlation analysis to analyze the linear correlation between different quality indicators, and use interactive design to allow users to click, filter and zoom to dynamically explore and analyze visual charts; During data transmission, data is encrypted using a symmetric encryption algorithm. Sensitive data is encrypted when stored. In the system, user requests are checked for permissions through middleware or decorators.

8. The group quality management system based on dynamic data source according to claim 1 is characterized in that: The data is classified and stored, each quality data is backed up regularly and a recovery function is provided, including: Deduplication is performed through a hash algorithm, and data from different data sources is integrated and cleaned before storage. The table structure is designed based on data characteristics and access patterns. A distributed database is used for large-scale dynamic data. Full data backup is performed regularly, and incremental backup is performed at preset time intervals.

9. The group quality management system based on dynamic data source according to claim 1 is characterized in that: The quality control plan formulation, quality warning and exception handling and decision support include: Based on historical quality data and business needs, a quality prediction model is constructed using a neural network algorithm, a quality classification model is constructed using a support vector machine algorithm, and a quality optimization model is constructed using a genetic algorithm. The quality prediction model, quality classification model, and quality optimization model are trained and optimized using the collected data to adjust the model parameters; Collect internal and external quality data, merge and associate data from different sources, integrate data based on common key fields, analyze the integrated data through association rule mining technology, determine the reasonable inspection frequency based on the ATI model, conduct inspections based on systematic sampling, and monitor quality data and process performance data in real time through process capability index; Based on the failure mode and effect analysis method, the potential failure modes, causes and impacts of quality problems are identified, the change indicators of RPN values ​​are tracked to regularly evaluate the effectiveness of risk response measures, and the knowledge base is obtained through historical data of neural network model training to provide suggestions and quality control strategies for decision support. Based on the results, interactive visualization charts are created to display key quality indicators and decision information in real time.

10. A group quality management method based on dynamic data source, characterized in that: Applied to the group quality management system based on dynamic data source as claimed in any one of claims 1 to 9, the group quality management method based on dynamic data source comprises the following steps: C1: Configure the data source, define the conditions and rules for switching the data source, use the default data source for data operations when the system cannot match the specific switching rules, and monitor the performance indicators of the connection status, data flow, and response time of each data source in real time; C2: Perform data entry and data verification, classify and store quality data according to different data sources, support retrieval of large amounts of quality data through vectorized indexing technology, and regularly back up quality data in each data source; C3: Extract quality data from multiple data sources, conduct comprehensive analysis and mining through a variety of data analysis tools and algorithms, generate quality statistical reports based on user needs, and support custom report formats; C4: Develop quality control plans based on quality objectives and data analysis results, associate quality control plans with corresponding data sources, set quality warning indicators and thresholds, record and track quality anomalies, support problem troubleshooting, cause analysis, and development and implementation of treatment measures, and provide decision support to management based on quality data analysis and quality control execution; C5: Group users for management, support adding, deleting, modifying and querying users, set different operation permissions for users and user groups based on data source access permissions, data entry permissions, data analysis permissions and report generation permissions, and record the system's operation logs, access logs and error logs.

Images