An operating management system maturity grading evaluation system based on an AI algorithm

Abstract

The application discloses a kind of based on AI algorithm's management system maturity grading evaluation system, belong to enterprise evaluation technical field, comprising: multi-format data input module, intelligent data preprocessing module, general maturity knowledge base module, AI core grading evaluation module, visual result output module, artificial review feedback module and model iteration optimization module;The application is based on the general evaluation model of five levels of maturity across system based on LPDCA logic, relies on the three-level progressive AI quantization algorithm of CNN-LSTM combined LoRA fine-tuning light-weight large model, and constructs the closed-loop intelligent agent architecture covering dynamic threshold adjustment, intelligent report template matching, multi-expert online review and monthly incremental training and double-weight reward reinforcement learning, realizes the unified quantization grading evaluation of whole management system, not only fills the blank of no standardization grading technology in the industry, eliminates subjective error of artificial review, improves evaluation efficiency, and can effectively reduce evaluation error.

Description

Technical Field

[0001] This invention belongs to the field of enterprise assessment technology, specifically relating to an AI-based maturity grading assessment system for business management systems. Background Technology

[0002] Currently, there is no unified global standard or quantitative assessment model for the maturity level of business management systems in the certification and auditing field. The evaluation of audit reports for various management systems such as ISO 9001, ISO 14001, and ISO 45001 all rely on the auditors' manual experience, only distinguishing between binary results of compliance and non-compliance. There is a lack of multi-level quantitative grading standards for maturity, and even more so, there is no unified evaluation benchmark across systems. As a result, each ISO standard system is judged independently, and the evaluation rules are fragmented, making it impossible to uniformly measure the overall management maturity of enterprises. Moreover, the method of judging by auditors' manual experience leads to a high degree of subjectivity. Different auditors may give the same report a maturity level that differs by more than two levels. In addition, the audit efficiency is low, the manual review time for a single full-element report is long, and the numerous sub-indicators of the system make it difficult for manual review to cover them all, resulting in large evaluation errors.

[0003] In addition, the current audit only lists the non-compliant items and cannot output phased improvement plans according to the maturity level. Furthermore, after the standard is updated, the manual evaluation rules are updated late, and it often takes more than several months to reach a unified audit consensus, which will reduce the accuracy of the evaluation. Summary of the Invention

[0004] The purpose of this invention is to provide an AI-based maturity grading assessment system for business management systems, in order to solve the problems encountered in the background art.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A maturity grading assessment system for business management systems based on AI algorithms, the system comprising:

[0007] The multi-format data input module is compatible with structured, semi-structured, and unstructured documents, automatically identifies various types of business management systems, and collects and audits raw data in batches, outputting raw audit datasets.

[0008] The intelligent data preprocessing module is used to receive the raw audit dataset, clean and organize the raw audit data, extract common maturity features and perform vector encoding to generate AI-readable feature data.

[0009] The general maturity knowledge base module is used to store the five-level maturity assessment criteria across systems, LPDCA grading weights, multi-business module mapping rules, industry benchmark library, and the mapping relationship between the clauses of each system and LPDCA dimensions, and output the grading assessment benchmark data;

[0010] The AI ​​core grading and assessment module is used to receive AI readable feature data and grading judgment benchmark data. It uses CNN-LSTM combined with LoRA to fine-tune a lightweight large model to complete the three-level maturity progressive calculation of clause level, LPDCA process level, and enterprise system level, and outputs the grading and assessment results.

[0011] The visualization results output module is used to receive the tiered assessment results, generate multi-format assessment reports, shortcomings descriptions and phased rectification and improvement plans based on the tiered assessment results, and output visualized assessment data.

[0012] The manual review and feedback module is used to manually review and correct the results of the hierarchical evaluation and the visual evaluation data under the preset hierarchical permission control mechanism, retain the manually corrected annotation samples, and output the review and correction data.

[0013] The model iteration and optimization module receives the review and correction data, performs incremental training based on the review and correction data, and uses a dual-weight reward function of matching score and professional score for reinforcement learning. It automatically iterates and updates the model parameters, outputs the optimized model parameters and sends them back to the AI ​​core grading and evaluation module to update the grading and evaluation strategy, forming a closed-loop self-optimization.

[0014] Furthermore, the multi-format data input module supports multimodal input of Excel / CSV / Word / PDF / image / audio, and the business management system types include ISO9001 / ISO14001 / ISO45001 / ISO50001 / ISO27001 systems;

[0015] Furthermore, the general maturity features in the intelligent data preprocessing module include five categories of general features: institutional integrity, implementation, risk control, monitoring and testing, and continuous improvement. Vector encoding is performed using a hybrid encoding method that combines 128-dimensional improved Word2Vec encoding with one-hot encoding.

[0016] Furthermore, the general maturity knowledge base module is deployed using a dual-database architecture, consisting of the Neo4j graph database and a MySQL relational database.

[0017] The cross-system five-level maturity assessment standard includes five levels: passive level, basic compliance level, standard control level, quantitative optimization level, and excellent level. It also introduces a dynamic threshold adjustment mechanism, which automatically optimizes the grading threshold range of each level based on historical assessment data of companies of similar size and industry in the industry benchmark library.

[0018] Furthermore, the basic model of the AI ​​core hierarchical evaluation module adopts a hybrid architecture that combines a three-layer CNN convolutional neural network, a two-layer LSTM long short-term memory network, a fully connected layer, and a LoRA-tuned lightweight LLaMA-7B large model; the model optimizer uses Adam, and the loss function uses cross-entropy.

[0019] The method for outputting the graded assessment results is as follows: score each clause at the clause level to obtain the clause-level score; weight the LPDCA process level according to the five dimensions of LPDCA to obtain the LPDCA level score; and perform a weighted conversion of the entire enterprise system at the enterprise system level to obtain the overall enterprise maturity score; based on the overall enterprise maturity score, output the graded assessment results.

[0020] Furthermore, the visualization output module is used to generate four types of data: PDF, Word, Excel, and ECharts dashboards. It also has a built-in intelligent report template engine that automatically matches the best display template based on the graded assessment results. For enterprises with high maturity levels, it highlights benchmark comparisons, while for enterprises with low maturity levels, it highlights the priority ranking of rectification.

[0021] Furthermore, the manual review and feedback module is configured with three levels of account permissions: administrator, senior auditor, and ordinary auditor. Only senior auditors can modify the permissions of the maturity level. When the same report is revised in opposite directions by different auditors, a multi-expert online review process is automatically triggered, and the review conclusions serve as positive and negative samples for model training.

[0022] Furthermore, the model iteration optimization module performs monthly incremental training with training parameters of batch=16 and 200 iterations; in the dual-weight reward function, the matching score weight is 0.6 and the professional score weight is 0.4; and a three-level early warning mechanism with accuracy rates of 95%, 90%, and 85% is set up, triggering an early warning and rectification when the accuracy rate of each level is lower than the corresponding threshold.

[0023] The beneficial effects of this invention are:

[0024] This invention utilizes a cross-system five-level maturity assessment model based on LPDCA logic, relying on a three-level progressive AI quantification algorithm combining CNN-LSTM with LoRA fine-tuning of a lightweight large model. It constructs a closed-loop intelligent agent architecture encompassing dynamic threshold adjustment, intelligent report template matching, multi-expert online review, monthly incremental training, and dual-weighted reward reinforcement learning. This achieves unified quantitative grading assessment of the entire business management system, filling the gap in the industry's lack of standardized grading technology, eliminating subjective errors in manual review, improving assessment efficiency, and effectively reducing assessment errors. Furthermore, by accurately identifying management shortcomings and automatically outputting phased rectification and improvement plans, the audit is upgraded from simple evaluation to management empowerment. The model's closed-loop self-iteration and three-level early warning mechanism ensure continuous system optimization and service reliability, guaranteeing automatic adaptation to new regulations without redevelopment.

[0025] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a system module diagram of the present invention;

[0028] Figure 2 This is a flowchart illustrating the reasoning process of the AI ​​core hierarchical evaluation module of this invention.

[0029] Figure 3 This is the overall flowchart of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] In one embodiment, an AI-based maturity grading assessment system for business management systems is disclosed, such as... Figure 1 , Figure 3 As shown, the system includes:

[0032] The multi-format data input module is compatible with structured, semi-structured, and unstructured documents. It automatically identifies various business management system types, collects and audits raw data in batches, and outputs raw audit datasets. The multi-format data input module supports multimodal input in Excel, CSV, Word, PDF, image, and audio formats. Business management system types include ISO 9001, ISO 14001, ISO 45001, ISO 50001, and ISO 27001 systems.

[0033] The multi-format data input module is the data entry point of this system, responsible for being compatible with multiple data formats, automatically identifying the type of business management system, and batch collecting and auditing raw data; specifically:

[0034] In terms of compatibility, the multi-format data input module supports data sources including: audit report text (including audit plans, audit checklists, audit records, non-conformity reports, audit summary reports, etc.), on-site audit evidence (including photos, videos, audio recordings, etc.), enterprise management ledger records (including quality records, environmental records, safety records, energy records, information security records, etc.), target performance data (including KPI achievement status, management review inputs, etc.), and historical rectification data (including corrective action reports, preventive action reports, continuous improvement records, etc.). Regarding file formats, it supports structured files (such as Excel, CSV), semi-structured files (such as XML, JSON), and unstructured files (such as Word, PDF, scanned images, on-site audio). Specifically: Excel / CSV files are read using the pandas library, parsing the table structure and extracting cell data; Word files are parsed using the python-docx library, extracting paragraph text, table content, and header / footer information; PDF files are parsed using the PyPDF2 or pdfplumber library, extracting text content and table structure; and image files are read using the PIL library, calling an OCR engine (such as Tesseract or a commercial OCR). The module performs text recognition using APIs; audio files are transcribed into speech using speech recognition engines such as Whisper or Baidu Speech Recognition API; the maximum batch size for this module is 50 files, with a total size not exceeding 1GB, to meet the needs of enterprises for batch review; in terms of parsing accuracy, the accuracy rate for table parsing is no less than 99%, the accuracy rate for image OCR recognition is no less than 98%, and the accuracy rate for speech transcription is no less than 97%.

[0035] In automatically identifying various types of business management systems, this module has a built-in business management system type identification engine, which can automatically identify the system type without manual selection; the identification logic is as follows:

[0036] First, initial screening is performed through keyword matching. The system has a built-in core keyword library for various ISO standards. For example, keywords for ISO 9001 include quality policy, customer satisfaction, process approach, PDCA, etc.; keywords for ISO 14001 include environmental aspects, compliance obligations, life cycle, environmental performance, etc.; keywords for ISO 45001 include hazard sources, risk classification and control, occupational health, safety performance, etc.; keywords for ISO 50001 include energy review, energy benchmarks, energy performance parameters, etc.; and keywords for ISO 27001 include information security policy, risk assessment, asset inventory, access control, etc. Second, confirmation is performed through clause numbering patterns. Each ISO standard has specific patterned clause numbering. For example, ISO 9001 uses a numbering pattern of 4 (organizational context), 5 (leadership role), 6 (planning), etc.; ISO 14001 uses a similar but different clause structure. The system uses regular expressions to match clause numbers to further confirm the system type. Finally, decision-making is achieved through multi-evidence fusion. When the keyword matching and clause number identification results are inconsistent, the system uses a voting mechanism to give a final identification result by integrating multiple evidence sources. When the identification confidence level is lower than a preset threshold, such as 85%, a manual confirmation prompt is triggered.

[0037] After identification, the module encapsulates all types of raw data into a raw audit dataset and outputs it to the intelligent data preprocessing module. The raw audit dataset is in JSON format and contains the following fields: file identifier (file_id), file type (file_type), system type (system_type), raw content (raw_content), collection timestamp (timestamp), and file source (source).

[0038] The intelligent data preprocessing module receives the original audit dataset, cleans and organizes the original audit data, extracts general maturity features and performs vector encoding to generate AI-readable feature data. The general maturity features in the intelligent data preprocessing module include five categories of general features: system integrity, implementation, risk control, monitoring and testing, and continuous improvement. Vector encoding is performed using a hybrid encoding method that combines 128-dimensional improved Word2Vec encoding with one-hot encoding.

[0039] The intelligent data preprocessing module is the data processing center of this system, responsible for cleaning, normalizing, extracting features, and vector encoding the raw audit data to generate standardized feature data that is readable by AI; specifically:

[0040] Data cleaning includes the following steps: First, noise removal: Automatically identify and remove irrelevant noise such as headers, page numbers, document attribute information, revision marks, and comments; match header and footer features using regular expressions (such as repetitive patterns like "Page X of Y" and "Review Report") and identify comments and revision marks through document structure analysis; Second, garbled text handling: Detect and repair garbled text caused by encoding issues, supporting automatic identification and conversion of common encodings such as UTF-8, GBK, GB2312, and ISO-8859-1. For garbled text segments that cannot be repaired, mark them as unrecognizable. And log the process; Third, deduplication: calculate the similarity between text segments. When the similarity exceeds a set threshold, such as 95%, it is judged as duplicate content. Only the earliest occurrence is retained, and the rest are automatically deduplicated. The similarity calculation adopts a dual judgment mechanism combining the Levenshtein algorithm based on edit distance and the cosine similarity based on semantic vectors; Fourth, missing value handling: perform missing value detection on the cleaned data and fill in the missing values ​​through interpolation, mean imputation, mode imputation, or context-based intelligent inference, etc., to ensure that the field missing rate after data cleaning does not exceed 5%;

[0041] The cleaned data needs to be uniformly converted into a standardized four-element field structure: [Clause Number - Evidence Content - LPDCA Attribution - Compliance Status]. Specifically, the clause number is matched with the clause numbering patterns of various ISO standards (such as 6.2, 7.1.5, etc.) using regular expressions and mapped to the standard clause numbers in the standard clause library; the evidence content extracts the specific audit evidence text corresponding to the clause, including document descriptions, record content, on-site observations, etc.; the LPDCA attribution automatically determines which dimension (L (Leadership), P (Planning), D (Implementation), C (Inspection), or A (Improvement)) the clause belongs to based on the mapping relationship between the clause number and LPDCA dimensions; the compliance status is initially determined based on keywords and semantic analysis in the evidence content (compliant / non-compliant / observation item).

[0042] Extracting general maturity features involves extracting five categories of general maturity features from structured data. The first category is institutional completeness features, including: institutional document coverage (number of established institutional documents / number of institutional documents that should be established), institutional hierarchy completeness (whether there are four levels of documents: policies, procedures, work instructions, and record forms), institutional approval process standardization (whether it has gone through the processes of drafting, reviewing, approving, issuing, and being controlled), institutional version management effectiveness (version number, revision history, obsolescence mark), institutional responsibility clarity (whether the responsible department / position is clearly defined), institutional interface coordination (the connection between related institutional documents), consistency between institutional documents and national / industry standards, and institutional periodic review mechanism (whether a periodic review cycle is stipulated). The second category is implementation and implementation. The characteristics include: completeness of execution records (actual number of execution records / number of records to be executed), compliance with execution frequency (actual execution frequency / frequency stipulated by regulations), compliance of execution personnel qualifications (certification rate), standardization of execution process (whether the procedures and steps are followed), achievement rate of execution results (the proportion of execution results that meet the target requirements), timeliness of handling execution anomalies (the time interval between anomaly discovery and handling), adequacy of execution resource allocation (whether personnel, equipment, and funds are sufficient), and consistency between execution site and documents (whether the actual on-site operation is consistent with the document stipulations), etc.; the third category is risk control characteristics, including: comprehensiveness of risk identification (number of identified risks / number of risks to be identified), accuracy of risk assessment (whether the risk level determination is reasonable), risk... The effectiveness of response measures (the degree of risk reduction after implementation), the soundness of the risk monitoring mechanism (whether there are regular risk reviews), the timeliness of risk warnings (whether warnings are received before risks occur), the completeness of risk emergency plans (whether there are plans and drill records), the balance between risks and opportunities (whether opportunities are identified and measures are formulated simultaneously), and the effectiveness of risk information communication (whether risk information is transmitted to relevant parties); the fourth category is the actual monitoring characteristics, including: the rationality of monitoring indicator settings (whether the indicators are measurable and related to the objectives), the completeness of monitoring data collection (data collection coverage), the depth of monitoring data analysis (whether trend analysis and comparative analysis are conducted), the timeliness of monitoring result reporting (whether the reporting cycle meets the requirements), and the closed loop of monitoring problem discovery. The five categories are: rate (the ratio from problem discovery to closure), effectiveness of monitoring equipment calibration (whether the measuring equipment is within its validity period), and compliance of monitoring personnel capabilities (whether the monitoring personnel possess the corresponding capabilities); the fifth category is continuous improvement characteristics, including: initiative in identifying improvement opportunities (whether improvement opportunities are actively sought rather than passively responded to), adequacy of improvement measure planning (whether improvement goals, measures, resources, and time limits are clearly defined), completion rate of improvement measure implementation (planned number of improvements / actual number completed), effectiveness of improvement effect verification (comparison of data before and after improvement), scope of improvement experience promotion (whether it is horizontally promoted to similar processes), degree of integration of improvement and innovation (whether new technologies / methods are introduced), and the driving role of management review in improvement (whether management review promotes systemic improvement), etc.Finally, these features are vector-encoded to generate AI-readable feature vectors; this module employs 128-dimensional improved Word2Vec encoding and one-hot encoding. This approach combines two encoding methods: Word2Vec and JSON. In the Word2Vec encoding part, a pre-trained Word2Vec model (trained on 12,000+ audit reports across 6 industries) encodes textual features (such as policy descriptions and execution records) into 128-dimensional dense vectors. Simultaneously, for audit domain terminology, domain-adaptive training is employed, incrementally training audit domain vocabulary on a general corpus to improve the vector representation quality of terminology. An attention mechanism is introduced to assign higher weights to keywords (such as "must," "should," "ensure," and "regularly"), enhancing semantic discriminability. In the one-hot encoding part, categorical features (such as compliance status: compliant / non-compliant / observation; LPDCA dimensions: L / P / D / C / A; system type: ISO9001 / 14001 / 45001 / 50001 / 27001, etc.) are one-hot encoded to generate sparse vectors. Finally, the Word2Vec dense vectors and the one-hot encoded sparse vectors are concatenated to form a hybrid feature vector, outputting AI-readable feature data in JSON format. ;

[0043] This approach involves multi-dimensional cleaning, unified structural transformation, and extraction of five major feature categories from the original review data. A hybrid encoding method combining 128-dimensional improved Word2Vec encoding and one-hot encoding is then used to generate AI-readable feature vectors. This achieves standardized, structured, and vectorized processing of multi-source heterogeneous review data, effectively solving problems such as complex data formats, subjective and arbitrary feature extraction, and missing key information in traditional manual review. It provides high-quality, highly consistent, and highly complete standardized input data for the subsequent AI core grading assessment module, significantly improving the accuracy and reliability of the entire assessment process.

[0044] The general maturity knowledge base module is used to store the five-level maturity judgment criteria across systems, LPDCA grading weights, multi-business module mapping rules, industry benchmark library, and the mapping relationship between the clauses of each system and LPDCA dimensions, and outputs the grading judgment benchmark data. The general maturity knowledge base module is deployed with a dual database of Neo4j graph database and MySQL relational database.

[0045] The cross-system five-level maturity assessment standard includes five levels: passive level, basic compliance level, standard control level, quantitative optimization level, and excellent level. It also introduces a dynamic threshold adjustment mechanism, which automatically optimizes the grading threshold range of each level based on historical assessment data of companies of similar size and industry in the industry benchmark library.

[0046] The general maturity knowledge base module is the knowledge hub of this system. It stores cross-system five-level maturity assessment standards, LPDCA grading weights, multi-business module mapping rules, industry benchmark libraries, and the mapping relationship between various system clauses and LPDCA dimensions, providing benchmark data for the AI ​​core grading assessment module; specifically:

[0047] The general maturity knowledge base module is deployed using a dual-database setup: Neo4j graph database and MySQL relational database. Neo4j graph database stores complex relationships between clauses, dimensions, and modules. With nodes and relationships as its core data model, graph databases are naturally suited for expressing network relationships such as clause-belong-dimension, dimension-include-module, and module-related-benchmark cases. Specific node types include: standard clause nodes (storing clause number, clause name, clause description, and applicable standard), LPDCA dimension nodes (storing dimension name, dimension definition, and dimension weight), business module nodes (storing module name, module definition, and module percentage), and benchmark case nodes (storing case number, company information, and industry classification). The database consists of several relational databases: scale level, assessment results, etc. Relationship types include: BELONGS_TO (clause belongs to dimension), CONTAINS (dimension contains modules), ASSOCIATED_WITH (module is associated with cases), SIMILAR_TO (cases are similar), etc. A MySQL relational database is used to store structured data such as grading thresholds, weight parameters, and historical assessment records. The main data tables include: Grading Threshold Table (stores score ranges for each grade and dynamic adjustment records), Weight Parameter Table (stores LPDCA fixed weights and industry fine-tuning parameters), Assessment Record Table (stores assessment results, assessment time, and assessment objects), and User Permission Table (stores user accounts, roles, and permission configurations), etc. Data consistency is maintained between the two databases through a data synchronization mechanism.

[0048] This knowledge base incorporates a cross-system five-level maturity assessment standard, dividing enterprise management maturity into five levels, such as... Figure 2 As shown, the first level is the passive level (score range: less than 60), characterized by: fragmented systems, reactive firefighting, and lack of standardized processes; specifically, the management system has not been established or is extremely incomplete, relying mainly on personal experience rather than systems and processes for management; responding passively only after problems occur, lacking preventive mechanisms; lacking clear management goals and performance indicators; and auditing evidence is mostly verbal descriptions or temporary compilations, lacking systematic records.

[0049] Level 2 is Basic Compliance (score range: 60-69). Characteristics: Basic documents are in place, meeting the minimum legal compliance requirements; specifically: basic management system documents have been established, covering the minimum clauses required by the standard; basic execution records exist, but are incomplete and non-standard; major risks can be identified and basic response measures can be developed; management objectives are initially set but lack systematicity; external audits have been passed, but many non-compliance items remain.

[0050] Level 3: Standard Control Level (Score Range: 70-79), Characteristic Description: Standardized processes and controllable indicators, reaching the mainstream industry level; Specifically, this is manifested in: complete management system documents covering all standard clauses; standardized process control and complete and standardized execution records; systematic risk control, capable of risk assessment and hierarchical management; reasonable performance indicator settings and routine data collection and analysis; ability to identify improvement opportunities and implement improvement measures; and effective management review promoting system operation.

[0051] Level 4 is the Quantitative Optimization Level (score range: 80-89), characterized by: quantitative data control, risk warning, and routine improvement; specifically manifested as: comprehensive data-driven management processes and real-time monitoring of key indicators; establishment of data analysis models capable of predicting trends and warning of risks; improvement measures based on data analysis rather than experience-based judgment; routine benchmarking management, aligning with industry best practices; effective knowledge management, systematic accumulation and reuse of lessons learned; and stable management performance reaching industry-leading levels.

[0052] Level 5 is the Excellent Level (score range: 90-100), characterized by: strategy-driven, management culture implemented, and continuous optimization for excellence; specifically manifested as: deep integration of management system and strategic goals, with management driving strategic achievement; management culture deeply ingrained, with all employees participating in continuous improvement; innovation becoming the norm, and management methods continuously innovating; performance reaching international leading levels, becoming an industry benchmark; proactive participation in standard setting, outputting best practices; and social responsibility and sustainable development fully integrated into management.

[0053] Simultaneously, a dynamic threshold adjustment mechanism is introduced. Based on historical assessment data of companies of similar size and industry in the industry benchmark database, the threshold range for each level is automatically optimized. The logic of dynamic adjustment is as follows: regularly (e.g., quarterly) analyze newly added assessment data in the industry benchmark database and calculate the maturity score distribution of companies of different industries and sizes; when the overall maturity level of companies in a certain industry improves, the grading threshold for that industry is adjusted upward accordingly (e.g., adjusting the lower limit of the third level from 70 to 71) to ensure the differentiation and incentive effect of the grading; when a major change occurs in a certain industry (e.g., the release of new standards or the popularization of new technologies), an emergency threshold adjustment process is triggered, which is implemented after review by the expert committee; the specific calculation formula for dynamic threshold adjustment is: new threshold = original threshold + α × (change in industry average score), where α is the adjustment coefficient (default 0.5), with a value range of 0.3-0.7, set by the system administrator according to industry stability.

[0054] The fixed weights for the five dimensions of LPDCA are: L (Leadership) 0.25, P (Planning) 0.20, D (Doing) 0.20, C (Checking) 0.20, and A (Improving) 0.15. This weight allocation is based on management practice research. For example, leadership is the primary driving force for the effective operation of a management system, so it has the highest weight. Planning, Doing, and Checking are the core links in the management process, so they have equal weights. Improving is the closed-loop link for management enhancement, so it has a slightly lower weight but is indispensable. At the same time, the weights can be slightly adjusted by ±0.05 according to the industry. For example, for highly regulated industries (such as pharmaceuticals and food), the weight of Planning (P) can be increased to 0.25, and the weight of Leadership (L) can be correspondingly decreased to 0.20. For innovation-driven industries (such as the Internet and high-tech industries), the weight of Planning (P) can be increased to 0.25, and the weight of Leadership (L) can be correspondingly decreased to 0.20. The weight of Improvement (A) can be increased to 0.20, and the weight of Inspection (C) can be decreased accordingly to 0.15. The mapping rules for each business module cover the main areas of enterprise operation and management, including: strategic management, organizational governance, leadership role, risk management, goal management, resource management, human resource management, infrastructure management, process operation control, product / service design and development, external process control, identification and traceability, customer satisfaction measurement, internal audit, management review, nonconforming output control, data analysis and evaluation, continuous improvement, corrective action, preventive action, knowledge management, innovation management, etc. The attribution and proportion of each business module in the five dimensions of LPDCA are defined by the mapping table in the knowledge base to ensure the comprehensiveness and systematicness of the assessment.

[0055] Meanwhile, the industry benchmark database contains over a thousand benchmark cases, covering six major industries: manufacturing, construction, services, energy, information technology, and healthcare. Each industry is categorized and stored according to company size (large, medium, and small). Benchmark cases include basic company information, assessment reports, maturity scores, scores for each dimension, and before-and-after comparisons of improvements. The knowledge base automatically crawls the latest regulations, including national standards, ISO international standards, and industry norms, daily, extracts change clauses using natural language processing technology, generates update summaries, and submits them to the expert committee for review before being added to the database. The automatic crawling sources include: the official website of the Standardization Administration of China, the official website of ISO, websites of various industry associations, and announcements from authoritative certification bodies.

[0056] This approach establishes a unified evaluation benchmark for the entire product category management system, using LPDCA's full-process management logic and multiple business modules as the underlying framework. This enables comparability and evaluation of different ISO systems within the same coordinate system, filling the gap in the industry's lack of standardized grading technology. A dual-database deployment architecture using Neo4j graph database and MySQL relational database fully leverages the relational expression advantages of graph databases and the structured data management advantages of relational databases, ensuring the integrity of knowledge storage and the efficiency of queries. A dynamic threshold adjustment mechanism is introduced to automatically optimize the grading threshold range for each level based on historical evaluation data from companies of similar size and industry in the industry benchmark database. This avoids evaluation distortion caused by standard rigidity, ensuring that evaluation results always have differentiation and incentive effects, and promoting the continuous improvement of the overall management level of the industry. Simultaneously, by automatically capturing the latest regulations daily and submitting them for expert review and database entry, the knowledge base is continuously updated and synchronized with standards, solving the industry problem of lagging updates to manual evaluation rules after the introduction of new regulations.

[0057] The AI ​​core grading and assessment module receives AI-readable feature data and grading benchmark data. It uses a CNN-LSTM combined with a LoRA-tuned lightweight large model to perform three levels of maturity progression calculation: clause-level, LPDCA process-level, and enterprise system-level, and outputs the grading and assessment results. The basic model of the AI ​​core grading and assessment module adopts a hybrid architecture that combines a three-layer CNN convolutional neural network, a two-layer LSTM long short-term memory network, a fully connected layer, and a LoRA-tuned lightweight LLaMA-7B large model. The model optimizer uses Adam, and the loss function uses cross-entropy.

[0058] The method for outputting the graded assessment results is as follows: score each clause at the clause level to obtain the clause-level score; weight the LPDCA process level according to the five dimensions of LPDCA to obtain the LPDCA level score; and perform a weighted conversion of the entire enterprise system at the enterprise system level to obtain the overall enterprise maturity score; based on the overall enterprise maturity score, output the graded assessment results.

[0059] The AI ​​core grading and assessment module is the intelligent core of this system. It receives AI-readable feature data and grading benchmark data, and uses a hybrid architecture combining CNN-LSTM with LoRA fine-tuning of a lightweight large model to complete three levels of maturity progression calculation: clause-level, LPDCA process-level, and enterprise system-level, outputting the grading and assessment results; specifically:

[0060] This module's basic model employs a hybrid architecture combining a three-layer CNN convolutional neural network, a two-layer LSTM long short-term memory network, fully connected layers, and a LoRA (Low-Rank Adaptation) fine-tuned lightweight LLaMA-7B large model. The CNN convolutional neural network is responsible for extracting local feature patterns from the input feature vector. The three convolutional layers are configured as follows: the first layer has a 3×1 kernel size, 64 channels, ReLU activation function, and a 2×1 pooling window; the second layer has a 3×1 kernel size, 128 channels, ReLU activation function, and a 2×1 pooling window; the third layer has a 3×1 kernel size, 256 channels, ReLU activation function, and global average pooling. The advantage of CNNs lies in their ability to capture local correlations in feature vectors, such as the association patterns between adjacent feature indicators. The LSTM long short-term memory network is responsible for modeling the temporal dependencies in the feature sequence. The two LSTM layers are configured as follows: the first layer has a hidden unit number... 128, return sequence; the second layer has 64 hidden units, return the final state; LSTM controls information flow through gating mechanisms (input gate, forget gate, output gate), effectively handling long sequence dependencies, suitable for modeling temporal features in audit evidence (such as the implementation sequence of rectification measures, the periodicity of management reviews, etc.); Fully connected layer: the features extracted by CNN and LSTM are fused and mapped to the maturity score space through two fully connected layers; the first fully connected layer has 256 neurons, the activation function is ReLU, and the dropout rate is 0.3; the second fully connected layer has 5 neurons (corresponding to five levels of maturity), the activation function is Softmax, and the output is the probability distribution of each level; LoRA fine-tuning lightweight LLaMA-7B part: for the professional semantic understanding needs of the audit domain, LLaMA-7B (Large) is introduced. The Language Model Meta AI (7B parameter version) is a large language model that uses LoRA technology for lightweight fine-tuning. The core idea of ​​LoRA is to freeze all parameters of the pre-trained large model and inject a low-rank matrix (low-rank decomposition into matrix A and matrix B, A∈R^{r×d}, B∈R^{d×r}, where r is the rank, usually 8 or 16) only into the attention module of the Transformer layer. Domain adaptation is achieved by training the low-rank matrix. Compared with full parameter fine-tuning, LoRA reduces the trainable parameters from 7 billion to millions (about 0.1% of the original parameters), improving training efficiency by more than 100 times while maintaining model performance. LLaMA-7B is responsible for understanding the complex semantics in the evidence, reasoning about implicit information, and generating evaluation reasons, making up for the shortcomings of traditional deep learning models in semantic understanding. Model fusion mechanism: the CNN-LSTM branch outputs a quantitative score based on structured features, and the LLaMA-LoRA branch outputs a qualitative evaluation based on semantic understanding. The results of the two branches are fused by weighting (quantitative score weight 0.7, qualitative evaluation weight 0).3) Obtain the final item-level score.

[0061] The model optimizer uses Adam (Adaptive Moment Estimation), which has the advantage of adaptively adjusting the learning rate and is suitable for handling sparse gradients and non-stationary objectives. Adam's hyperparameters are set as follows: initial learning rate 0.001, β1=0.9, β2=0.999, ε=1e-8, and weight decay 1e-5. The loss function is cross-entropy loss, used to measure the difference between the model's output rank probability distribution and the true label. For sample i, the true label is... (One-hot encoding), the model predicts the probability as Then the cross-entropy loss is: ,in, The number of maturity level categories is 5 in this example, corresponding to five major categories. Let be the true label value of the i-th sample in the j-th category. To predict the probability that the i-th sample belongs to the j-th class, the total loss is the average of the losses of all samples, plus an L2 regularization term to prevent overfitting. The training dataset consists of over 12,000 multi-system labeled reports from six major industries (manufacturing, construction, services, energy, information technology, and healthcare), manually labeled by a team of senior auditors. The annotations include clause-level maturity scores, LPDCA dimension scores, and system-level maturity levels. The dataset is divided into training, validation, and test sets in a 7:2:1 ratio. Initial training parameters: batch size = 32, basic iterations 1000 rounds; early stopping mechanism is used during training. Stopping: Automatically stop training and save the optimal model when the validation set loss no longer decreases for 10 consecutive rounds; Learning rate decay strategy: Multiply the learning rate by 0.5 every 200 rounds to ensure convergence stability; Model performance metrics: Single sample inference time is less than 300 milliseconds, meeting real-time evaluation requirements; Item-level evaluation accuracy is no less than 92%, LPDCA-level evaluation accuracy is no less than 95%, and system-level evaluation accuracy is no less than 97%.

[0062] A three-tiered maturity calculation approach is adopted: clause-level, LPDCA process-level, and enterprise system-level. The output is a graded assessment result, with the first level being clause-level assessment. For each clause's corresponding audit evidence, the AI ​​model independently evaluates the maturity score and outputs the clause's maturity score. The assessment method involves inputting the clause's feature vector into a CNN-LSTM-LLaMA hybrid model. The model outputs a five-level probability distribution, and the expected score is calculated through weighted summation: [Expression follows]. Where F is the expected maturity score of the single-challenge clause, k is the maturity level, corresponding to five levels: passive, basic compliance, standard control, quantitative optimization, and excellent; P is the probability that the model predicts the clause belongs to the kth level; at the same time, the model outputs the assessment confidence level (0~100%) of the clause, and clauses with a confidence level below 70% are marked as needing review.

[0063] The second level is the LPDCA process-level assessment; based on the clause-level scores, a weighted aggregation is performed according to the five dimensions of LPDCA to obtain the maturity score for each dimension. The specific calculation formula is as follows: ,in, This is a weighted maturity score for a specific LPDCA dimension (one of L / P / D / C / A), where D is the set of terms belonging to that dimension. The clause-level score for the i-th clause is output by the first-level evaluation. The weight of the i-th clause is determined by its importance in the standard, with core clauses having higher weights and general clauses having lower weights. Then, the process-level comprehensive score is calculated using the LPDCA fixed weight: LPDCA score = 0.25 × L score + 0.20 × P score + 0.20 × D score + 0.20 × C score + 0.15 × A score.

[0064] The third level is the enterprise system-level assessment; it combines the clause-level and LPDCA-level scores to calculate the enterprise's overall maturity score. The calculation formula is: System-level score = Clause mean × 0.4 + LPDCA mean × 0.6; where the clause mean is the arithmetic mean of all clause scores, and the LPDCA mean is the weighted average of the five dimensions; the clause mean reflects the breadth of management (how many clauses are covered), and the LPDCA mean reflects the depth of management (the balance of each management dimension). The combination of the two comprehensively reflects the enterprise's overall maturity. In addition, a major non-compliance downgrade mechanism is adopted: when the system detects that the enterprise has major non-compliance items, a downgrade mechanism is adopted. In cases of systemic failures, violations of laws and regulations, or major safety hazards, the score is immediately downgraded by one level, regardless of the calculated score. The criteria for determining major non-compliance items are defined by the list of major non-compliance clauses in the knowledge base, covering the core clauses in various ISO standards (such as ISO 9001's 8.5.1 control of production and service provision, ISO 45001's 8.1.3 management changes, etc.). Finally, when the system-level score falls into the passive level range, it is level 1; when it falls into the basic compliance level range, it is level 2; when it falls into the standard control level range, it is level 3; when it falls into the quantitative optimization level range, it is level 4; and when it falls into the excellent level range, it is level 5.

[0065] In this approach, the solution constructs a hybrid architecture combining CNN-LSTM and LoRA-tuned lightweight LLaMA-7B, and designs a three-level progressive evaluation logic from clause level to LPDCA process level to enterprise system level. It weights and fuses the quantitative scoring of the CNN-LSTM branch with the semantic understanding of the LLaMA-LoRA branch, and introduces a major non-compliance downgrade mechanism. This achieves accurate quantitative evaluation of audit evidence through a progressive and hierarchical aggregation from micro-clauses to macro-systems, significantly reducing the time consumption of single-sample reasoning and significantly improving the accuracy of system-level evaluation. It also takes into account the quantitative scoring capability of structured features and the ability to understand and reason about complex semantics, effectively eliminating the subjective errors of traditional manual review and significantly improving the evaluation efficiency and reliability of results.

[0066] The visualization results output module receives the tiered assessment results, generates multi-format assessment reports, shortcomings descriptions, and phased rectification and improvement plans based on the results, and outputs visualized assessment data. This module also generates four types of data: PDF, Word, Excel, and ECharts dashboards. It has a built-in intelligent report template engine that automatically matches the best display template based on the tiered assessment results, highlighting benchmark comparisons for high-maturity enterprises and prioritizing rectification for low-maturity enterprises.

[0067] The visualization results output module is responsible for receiving the tiered assessment results, generating multi-format assessment reports, shortcomings descriptions, and phased rectification and improvement plans, and outputting visualized assessment materials, specifically:

[0068] The output includes four core components: First, overall maturity level: clearly stating the company's overall maturity level (levels 1-5) and corresponding score, while also displaying the level and score for each LPDCA dimension, as well as a maturity radar chart for each business module; Second, LPDCA sub-item scores: displaying the scores, levels, comparisons with industry averages, and gaps with benchmark companies for the five dimensions of L, P, D, C, and A in tabular and bar chart formats; Third, detailed list of non-compliance points: listing all clauses in the clause-level assessment that scored below the basic compliance level, including clause number, clause name, current score, problem description, evidence summary, and risk level (high / medium / low); Fourth, phased maturity improvement plan: based on the weakness analysis, automatically generating a phased maturity improvement plan. A phased plan for upgrading from the current level to the next; the plan includes: phase goals (e.g., upgrading from level 3 to level 4), key tasks (items / dimensions requiring key improvement), responsible departments (automatically assigned based on item attribution), completion timeframe (automatically estimated based on task complexity, typically 3-6 months), resource requirements (human resources, funding, equipment), and acceptance criteria (what indicators are considered completion); output formats support four types of documents: formal PDF assessment report, editable Word report, Excel defect log, and ECharts interactive dashboard; the formal PDF assessment report uses standard A4 format and includes a cover (company name, assessment date, assessment agency, level identifier), table of contents, implementation summary, and detailed assessment results. Results, improvement suggestions, and appendices (evidence list, methodology descriptions); each report can be generated in no more than 10 seconds. Word editable reports are identical to PDFs in content but retain editable formatting, allowing companies to adjust improvement plans according to their own circumstances; each report can be generated in no more than 10 seconds. Excel defect logs list all nonconformities and observations in tabular form, supporting filtering, sorting, and exporting. Fields include: serial number, system type, clause number, problem description, severity, responsible department, planned completion date, actual completion date, and verification results; each report can be generated in no more than 5 seconds. ECharts interactive dashboards are web-based visual dashboards that support interactive viewing in a browser, including a maturity radar chart (displaying five...). The dashboard features a variety of charts, including dimensional scores, trend line charts (showing changes across assessments), a weakness heatmap (showing the score distribution of each module), and benchmarking bar charts (comparing with industry averages and benchmarks). The dashboard supports drill-down (from system level to dimension and clause level) and interactive elements (selecting a dimension simultaneously highlights other charts). Batch generation of 50 reports takes no more than 300 seconds. It also includes a built-in intelligent report template engine that automatically matches the best presentation template based on the tiered assessment results, enabling differentiated report generation. The template matching logic is as follows: when the enterprise maturity level is 4 or 5 (high maturity level), a benchmark comparison template is automatically matched. The report emphasizes comparative analysis with industry benchmarks, best practice extraction, and continuous innovation suggestions, while downplaying the listing of problems and strengthening the display of strengths.When the enterprise maturity level is Level 1 or 2 (low maturity level), the rectification priority template is automatically matched. The report highlights the priority ranking of rectification (ranked by risk level and impact), recommendations for quick-result measures, and suggestions for resource investment, strengthening problem orientation and providing actionable improvement steps. When the enterprise maturity level is Level 3 (standard control level), the balanced improvement template is automatically matched. The report takes into account both consolidating strengths and improving weaknesses, and provides the key path for leaping from Level 3 to Level 4.

[0069] In this way, this solution builds an intelligent report template engine that automatically matches differentiated display templates based on the company's maturity level. For high-maturity companies, it highlights benchmark comparisons and best practices; for low-maturity companies, it emphasizes rectification priorities and recommendations for quick-result measures. It also supports the generation of four types of documents: formal PDF assessment reports, editable Word reports, Excel defect ledgers, and ECharts interactive dashboards. It has the efficient output capability of generating a single document in no more than 10 seconds and generating 50 documents in batches in no more than 300 seconds. This achieves a paradigm upgrade from single-format output to differentiated, multi-dimensional, and interactive assessment presentation, enabling assessment reports to accurately adapt to the actual needs of companies at different maturity levels, and significantly improving the practicality, readability, and management empowerment value of the assessment results.

[0070] The manual review and feedback module is used to manually review and correct reports based on the tiered assessment results and visual assessment data under a preset hierarchical permission control mechanism. It retains manually corrected and annotated samples and outputs review and correction data. The manual review and feedback module has three levels of account permissions: administrator, senior auditor, and ordinary auditor. Only senior auditors can modify the permissions of the maturity level. When the same report is corrected in opposite directions by different auditors, a multi-expert online review process is automatically triggered, and the review conclusions serve as positive and negative samples for model training.

[0071] The manual review and feedback module is the human-machine collaborative node of this system. Under a preset hierarchical permission control mechanism, it performs manual review and correction based on the hierarchical evaluation results and visualized evaluation data, retains manually corrected annotation samples, and outputs review and correction data; specifically:

[0072] This module employs a tiered access control mechanism with three account levels: administrator, senior auditor, and regular auditor. Two-factor authentication (password + SMS verification code / hardware token) ensures account security. Administrators have full access to system functions, including user management, permission configuration, knowledge base maintenance, model parameter adjustment, and global data viewing. Administrators can view all assessment reports and review records but cannot modify assessment results (ensuring assessment independence). Senior auditors have permissions to review assessment reports, correct grades, initiate review meetings, and annotate samples. Senior auditors can modify the maturity level of the AI ​​assessment (±1 level), and must provide a reason for modification (selected from a preset reason library or custom input). Modification records are permanently archived. Senior auditors can also fine-tune the clause-level scores of the AI ​​assessment (±0.5 points) for... Calibrate model bias; Regular auditors have the following permissions: view assessment reports, add partial annotations, and submit comments. Regular auditors can view AI assessment results, add annotations to questionable clauses (such as insufficient evidence, misunderstanding, disputes over standard application, etc.), and submit them to senior auditors for review; regular auditors do not have the right to modify assessment results; when the same report is revised in opposite directions by different auditors (e.g., auditor A revised level 3 to level 4, auditor B revised level 3 to level 2), the system automatically triggers a multi-expert online review process and generates review questions (including report summary, AI assessment results, each auditor's revision comments and reasons), which are then pushed to the expert committee (composed of 5 or more senior auditors, excluding auditors who have already participated in the revision); experts independently review the report through the online review system and submit their respective judgments; the system adopts the Delphi method. The method involves multiple rounds of anonymous feedback until a consensus is reached (more than 2 / 3 of the experts agree). The review conclusion serves as the final evaluation result and is marked as a high-quality training sample. The review conclusion is also used as positive and negative samples for model training. If the review conclusion supports the original AI evaluation result, the sample is used as a correct positive sample to enhance the model's confidence. If the review conclusion corrects the AI ​​evaluation, the sample is used as an incorrect negative sample for incremental training to correct model bias. After manual review, the module outputs review and correction data, including: the original AI evaluation result, the manual correction content, the reason for the correction, the reviewer information, the correction timestamp, and the review record (if any). The review and correction data is output to the model iteration and optimization module in structured JSON format as a data source for incremental training.

[0073] The model iteration and optimization module receives the review and correction data, performs incremental training based on the data, and uses a dual-weight reward function of matching score and professional score for reinforcement learning. It automatically iterates and updates the model parameters, outputs the optimized model parameters, and sends them back to the AI ​​core grading and evaluation module to update the grading and evaluation strategy, forming a closed-loop self-optimization. The model iteration and optimization module performs monthly incremental training with batch=16 and 200 iterations. In the dual-weight reward function, the matching score has a weight of 0.6 and the professional score has a weight of 0.4. A three-level early warning mechanism with accuracy of 95%, 90%, and 85% is set up. When the grading accuracy falls below the corresponding threshold, an early warning and rectification are triggered.

[0074] The model iteration and optimization module is the self-evolution engine of this system. It receives the review and correction data, performs incremental training based on the data, and uses a dual-weight reward function of matching score and professional score for reinforcement learning. It automatically iterates and updates the model parameters, outputs the optimized model parameters, and sends them back to the AI ​​core grading and evaluation module to update the grading and evaluation strategy, forming a closed-loop self-optimization. Specifically:

[0075] This module performs incremental training at a fixed cycle, with a default cycle of monthly (automatically triggered at midnight on the 1st of each month). The training data source consists of all the review and correction data accumulated in the previous month, as well as newly added benchmark case data. Incremental training parameters: batch size=16 (smaller than the initial training size of 32, adapting to the small incremental data volume), 200 iterations; an incremental learning strategy is adopted: freezing the underlying parameters of CNN-LSTM (preserving the learned general feature extraction capabilities), and only fine-tuning the parameters of the top fully connected layer and LoRA low-rank matrix (adapting to newly added domain knowledge). This strategy avoids catastrophic forgetting. Forgetting, which means not losing previously acquired knowledge when learning new knowledge; the incremental training process is as follows: First, data preprocessing, converting the revised data into a feature vector format consistent with the initial training; Second, data augmentation, oversampling and noise injection (adding Gaussian noise to the feature vectors) on a small number of revised samples to expand the amount of training data; Third, model fine-tuning, loading the best model from last month as the initial weights and performing 200 rounds of training; Fourth, model validation, evaluating the model performance on the validation set, saving it as a new version if the accuracy improves, otherwise reverting to the version from last month; Fifth, model release, pushing the validated model parameters to the AI ​​core hierarchical evaluation module to update the online inference model;

[0076] A dual-weighted reward function is used for reinforcement learning to optimize the model's decision-making strategy. The reward function is defined as: Total Reward R = 0.6 × Matching Score + 0.4 × Professional Score. The Matching Score measures the consistency between the model's output and the human review result. The calculation formula is: Matching Score = 1 - |Model Output Level - Human Review Level| / 4. The Matching Score is 1.0 when the model output and human review are completely consistent; it is 0 when there is a difference of 4 levels (maximum difference). The Matching Score reflects the model's imitation ability, i.e., its ability to learn human expert judgment patterns. The Professional Score... The Score is used to measure the professional rationality of the model's output and is independently evaluated by an expert committee. Evaluation dimensions include: sufficiency of the evaluation rationale (whether there is sufficient evidence to support it), logical consistency (whether the evaluations of each clause are self-consistent), standard compliance (whether it conforms to the spirit of ISO standard provisions), and the feasibility of improvement suggestions. Each dimension has a maximum score of 1.0, and the professional score is the average of the four dimensions. The professional score reflects the model's professional level, that is, its ability to go beyond simple imitation and possess professional review competence. The matching score (0.6) has a higher weight than the professional score (0.4) because while human review results are authoritative, they may contain individual differences, and overfitting to human results may lead the model to inherit human biases. Maintaining a 0.4 weight for the professional score encourages the model to maintain independence in professional rationality, even when there are significant biases in human expert judgment, adhering to professional principles. This design balances the need to learn from human experience and maintain professional independence. The reinforcement learning algorithm uses PPO (Proximal Policy). The Optimization algorithm (proximal policy optimization) is a policy gradient reinforcement learning method proposed by OpenAI. It ensures training stability by limiting the policy update magnitude (pruning the objective function). The state space is the input feature vector, the action space is the probability distribution of five maturity levels, the policy network is a CNN-LSTM-LLaMA hybrid model, and the value network is an independent Critic network (evaluating the expected cumulative reward of the current state).

[0077] A three-tiered early warning mechanism is set up for accuracy rates of 95%, 90%, and 85%. When the accuracy rate falls below the corresponding threshold, an early warning and rectification measure is triggered. The specific warning levels are defined as follows: Green Warning (Accuracy ≥ 95%): Excellent model performance, normal iteration; Yellow Warning (90% ≤ Accuracy < 95%): Slight decline in model performance, triggering automatic diagnosis, analyzing the cause of the accuracy decline (e.g., changes in data distribution in a certain industry, updates to a certain standard clause, etc.), adjusting the training strategy; Red Warning (Accuracy < 90%): Significant decline in model performance, triggering emergency rectification, suspending automatic evaluation services, activating a backup model (last month's version or basic version), and simultaneously initiating expert committee intervention analysis; Black Warning (Accuracy < 85%): Severe model failure, triggering a system-level emergency response, full rollback. The system reverts to manual review mode, freezes model iterations, and allows the technical team and expert committee to jointly investigate root causes (such as data contamination, model overfitting, or significant changes in standards). Early warning monitoring indicators include graded accuracy (the proportion of system-level assessments consistent with manual review), feature matching degree (the matching rate between model-extracted features and expert-annotated features), and inference speed (average inference time per sample). These three indicators are monitored in real time, and triggering a threshold for any indicator initiates the corresponding early warning process. After an early warning is triggered, the system automatically sends notifications to the administrator and expert committee (email + SMS + system message). The notification includes: early warning level, triggering indicator, current value, threshold, trend analysis, and recommended measures. Yellow warnings require a response within 24 hours, red warnings within 4 hours, and black warnings within 1 hour.

[0078] In this way, this solution achieves a closed-loop self-optimization capability by constructing a monthly incremental training mechanism, a dual-weight reward reinforcement learning mechanism, and a three-level accuracy early warning mechanism. This capability allows the model to continuously learn new knowledge while maintaining the stability of old knowledge, achieving a balance between professional rationality and consistency with experience, and triggering safeguards in a timely manner when performance declines to ensure the continuous and reliable evaluation service. This enables the model to automatically iterate and update monthly and complete adaptation within 1-2 months after the introduction of new regulations, without the need for redevelopment each time. This fundamentally solves the industry problems of lagging updates to traditional manual evaluation rules and model solidification failure.

[0079] The above description is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined in the claims, they should all fall within the protection scope of the present invention.

Claims

1. A maturity grading assessment system for business management systems based on AI algorithms, characterized in that, The system includes: The multi-format data input module is compatible with structured, semi-structured, and unstructured documents, automatically identifies various types of business management systems, and collects and audits raw data in batches, outputting raw audit datasets. The intelligent data preprocessing module is used to receive the raw audit dataset, clean and organize the raw audit data, extract common maturity features and perform vector encoding to generate AI-readable feature data. The general maturity knowledge base module is used to store the five-level maturity assessment criteria across systems, LPDCA grading weights, multi-business module mapping rules, industry benchmark library, and the mapping relationship between the clauses of each system and LPDCA dimensions, and output the grading assessment benchmark data; The AI ​​core grading and assessment module is used to receive AI readable feature data and grading judgment benchmark data. It uses CNN-LSTM combined with LoRA to fine-tune a lightweight large model to complete the three-level maturity progressive calculation of clause level, LPDCA process level, and enterprise system level, and outputs the grading and assessment results. The visualization results output module is used to receive the tiered assessment results, generate multi-format assessment reports, shortcomings descriptions and phased rectification and improvement plans based on the tiered assessment results, and output visualized assessment data. The manual review and feedback module is used to manually review and correct the results of the hierarchical evaluation and the visual evaluation data under the preset hierarchical permission control mechanism, retain the manually corrected annotation samples, and output the review and correction data. The model iteration and optimization module receives the review and correction data, performs incremental training based on the review and correction data, and uses a dual-weight reward function of matching score and professional score for reinforcement learning. It automatically iterates and updates the model parameters, outputs the optimized model parameters and sends them back to the AI ​​core grading and evaluation module to update the grading and evaluation strategy, forming a closed-loop self-optimization.

2. The maturity grading assessment system for business management systems based on AI algorithms according to claim 1, characterized in that, The multi-format data input module supports multimodal input of Excel / CSV / Word / PDF / image / audio, and the business management system types include ISO9001 / ISO14001 / ISO45001 / ISO50001 / ISO27001.

3. The maturity grading assessment system for business management systems based on AI algorithms according to claim 1, characterized in that, The intelligent data preprocessing module includes five categories of general maturity features: institutional integrity, implementation, risk control, monitoring and testing, and continuous improvement. It employs a hybrid encoding method that combines 128-dimensional improved Word2Vec encoding with one-hot encoding for vector encoding.

4. The maturity grading assessment system for business management systems based on AI algorithms according to claim 1, characterized in that, The general maturity knowledge base module is deployed using a dual-database architecture, consisting of Neo4j graph database and MySQL relational database. The cross-system five-level maturity assessment standard includes five levels: passive level, basic compliance level, standard control level, quantitative optimization level, and excellent level. It also introduces a dynamic threshold adjustment mechanism, which automatically optimizes the grading threshold range of each level based on historical assessment data of companies of similar size and industry in the industry benchmark library.

5. The maturity grading assessment system for business management systems based on AI algorithms according to claim 1, characterized in that, The basic model of the AI ​​core hierarchical evaluation module adopts a hybrid architecture that combines a three-layer CNN convolutional neural network, a two-layer LSTM long short-term memory network, a fully connected layer, and a LoRA fine-tuned lightweight LLaMA-7B large model; the model optimizer uses Adam, and the loss function uses cross-entropy. The method for outputting the graded assessment results is as follows: score each clause at the clause level to obtain the clause-level score; weight the LPDCA process level according to the five dimensions of LPDCA to obtain the LPDCA level score; and perform a weighted conversion of the entire enterprise system at the enterprise system level to obtain the overall enterprise maturity score; based on the overall enterprise maturity score, output the graded assessment results.

6. The maturity grading assessment system for business management systems based on AI algorithms according to claim 1, characterized in that, The visualization output module generates four types of data: PDF, Word, Excel, and ECharts dashboards. It also has a built-in intelligent report template engine that automatically matches the best display template based on the grading assessment results. For enterprises with high maturity levels, it highlights benchmark comparisons, while for enterprises with low maturity levels, it emphasizes the priority ranking of rectification.

7. The maturity grading assessment system for business management systems based on AI algorithms according to claim 1, characterized in that, The manual review and feedback module has three levels of account permissions: administrator, senior auditor, and ordinary auditor. Only senior auditors can modify the permissions of the maturity level. When the same report is revised in opposite directions by different auditors, a multi-expert online review process is automatically triggered, and the review conclusions serve as positive and negative samples for model training.

8. The maturity grading assessment system for business management systems based on AI algorithms according to claim 1, characterized in that, The model iteration optimization module performs monthly incremental training with training parameters of batch=16 and 200 iterations. In the dual-weight reward function, the matching score weight is 0.6 and the professional score weight is 0.

4. A three-level early warning mechanism with accuracy of 95%, 90%, and 85% is set. When the accuracy of a level is lower than the corresponding threshold, an early warning and rectification are triggered.

Images