Dynamic performance evaluation index optimization system based on machine learning

Abstract

The invention provides a dynamic performance evaluation index optimization system based on machine learning, and belongs to the technical field of machine learning, and the system comprises a data acquisition module which is used for obtaining multi-source heterogeneous data in a layered manner, and carrying out the fine processing of a crawled file; the preprocessing module is used for performing fusion preprocessing on the multi-source heterogeneous data and the refined processing file, and constructing a mapping relation set of crawling semantic tags and business dimensions; the model construction module is used for learning the mapping relation set based on a machine learning technology and constructing and training a dynamic index model; and the thread optimization module is used for carrying out verification analysis on the dynamic index model, determining an optimization thread scheme for each dynamic index and carrying out self-adaptive optimization on the dynamic index model. Reliable support is provided for performance evaluation, and the fund configuration efficiency is optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a dynamic performance evaluation index optimization system based on machine learning. BACKGROUND

[0002] The existing research on the performance evaluation of vocational skill training is scattered, and there is no complete index system covering "input-output-benefit-satisfaction". For example, in most domestic areas, performance evaluation is only conducted through administrative assessment of the financial department, which cannot fully reflect the training quality and employment effect (such as the research and development data mentioned that "administrative evaluation is the main method, which cannot meet the management requirements"). Traditional performance evaluation indexes are mostly set statically, which is difficult to adapt to the update and data changes. For example, when the government or enterprises introduce new training subsidies (such as the requirement of "AI skill training"), the existing index system cannot automatically respond, leading to the disconnection between the crawling and the evaluation. Vocational skill training involves multi-source heterogeneous data such as finance, institution operation, and student employment, but the existing technology lacks efficient integration and semantic analysis of data. For example, the mapping relationship between the regulatory requirements in the crawling file (such as "funds performance audit") and the structured data (such as financial grant records) has not been effectively explored, leading to the deviation of index design and crawling intention.

[0003] Therefore, the present application proposes a dynamic performance evaluation index optimization system based on machine learning. SUMMARY

[0004] The present application provides a dynamic performance evaluation index optimization system based on machine learning to solve the technical problems mentioned above.

[0005] The present application proposes a dynamic performance evaluation index optimization system based on machine learning, which comprises: A data acquisition module is used to acquire multi-source heterogeneous data in layers, and to perform fine processing on the crawling files; A preprocessing module is used to fuse and preprocess the multi-source heterogeneous data and the fine processing files, and to construct a mapping relationship set between the crawling semantic labels and the business dimensions; A model construction module is used to learn the mapping relationship set based on machine learning technology, and to construct and train a dynamic index model; A thread optimization module is used to verify and analyze the dynamic index model and determine the optimization thread scheme for each dynamic index, and to perform adaptive optimization on the dynamic index model.

[0006] Preferably, the data acquisition module comprises: A first acquisition unit is used to acquire structured data in the field of vocational skill training; A second collection unit is configured to collect operation data of the training institution by using an Internet of Things sensor network, wherein the operation data is related to equipment utilization, teacher class hour proportion, and site idle rate; A third collection unit is configured to obtain employment data of the students by using a block chain storage platform. The structured data, the operation data, and the employment data constitute multi-source heterogeneous data.

[0007] Preferably, the data acquisition module further comprises: A fourth collection unit is configured to obtain crawling files of a publication data source by using a directional web crawler, wherein a counter- crawling strategy is configured for a counter- crawling mechanism of the crawling files, and data is obtained by simulating interaction through a headless browser. A file traversal unit is configured to traverse a mark tag in the crawling files, and replace the corresponding mark tag with a corresponding character according to a type of the mark tag, wherein the character is a non-encrypted character and an encrypted character. A format processing unit is configured to scan the replaced text, combine the continuous characters into an encrypted field, and perform position radiation on the encrypted field and the remaining text in the crawling text except the mark tag, to obtain an encrypted processing file, wherein the processing file is a fine processing result.

[0008] Preferably, the preprocessing module comprises: A data cleaning unit is configured to clean the multi-source heterogeneous data. A system construction unit is configured to define a crawling object set related to performance evaluation as a multi-level user portrait tag system for each object, wherein the multi-level user portrait tag system is composed of core tags, important tags, and general tags. A score determination unit is configured to extract text information from the processing file, and count the number of occurrences of a user portrait tag with a label level greater than or equal to a preset level for each object, to obtain a label matching degree score. A targeting determination unit is configured to determine an object with a label matching degree score greater than a preset score as a target crawling object of the processing file. An analysis unit is configured to perform hierarchical semantic analysis on the processing file according to the target crawling object, wherein the hierarchical semantic analysis is related to a supervision type semantic targeted by enterprise supervision and an operation type semantic targeted by the training institution. A multi-label classification unit is configured to perform multi-label classification on the hierarchical semantic analysis result by using a bidirectional encoding representation model, to output a crawling semantic label, and to establish a mapping relationship set with a business dimension of the cleaned structured data.

[0009] Preferably, the model construction module comprises: a strengthening processing unit, configured to perform strengthening processing on feature weights of core semantics associated with the targeted crawling object; a weakening processing unit, configured to count a strengthening weight amount of the targeted crawling object, and perform weakening processing on feature weights of core semantics associated with the non-targeted crawling object according to the strengthening weight amount; a model construction unit, configured to construct a dynamic index model based on the weakening processing result, the strengthening processing result, and the mapping relationship set.

[0010] Preferably, the model construction unit comprises: a structure construction sub-unit, configured to construct a text-label-association three-layer coding structure for the targeted crawling core semantics after the strengthening processing; a mechanism deployment sub-unit, configured to deploy a teacher-student feature distillation mechanism for the non-targeted semantic after the weakening processing; a constraint conversion sub-unit, configured to convert the crawling-business mapping relationship set into a graph structure constraint; a model sub-unit, configured to obtain a dynamic index model based on the graph structure constraint, the three-layer coding structure, and the graph structure constraint.

[0011] Preferably, the thread optimization module comprises: a calculation unit, configured to calculate a prediction accuracy and a recall rate of each core target in the dynamic index model by using hierarchical cross-validation, and introduce uncertainty quantification to output a confidence interval of a prediction result, and identify a difference between cognitive uncertainty and accidental uncertainty of the dynamic index model; a coverage determination unit, configured to construct a multi-dimensional knowledge graph, and calculate a coverage of the dynamic index model on a crawling clause by using a graph convolution network; a gap positioning unit, configured to position an index gap by using a semantic difference algorithm for a newly published crawling, and mark as a crawling mismatch point; a simulation unit, configured to simulate an extreme scenario impact, test sensitivity of the dynamic index, and calculate a coefficient of variation of an index in each window by using a sliding window verification to identify an index drift node; a scheme acquisition unit, configured to obtain an optimization thread scheme of each dynamic index from a resource-policy matching table of deep reinforcement learning based on the difference, the coverage, the crawling mismatch point, and the index drift node, the optimization thread scheme being related to an execution algorithm, a resource, a priority, and an execution timing of the dynamic index.

[0012] Preferably, nodes of the multi-dimensional knowledge graph are crawling clauses and evaluation indexes, and edges are semantic associations and constraint relationships.

[0013] Preferably, the dynamic indicators are static weight type indicators, dynamic feedback type indicators and new scene indicators of the dynamic indicator model.

[0014] Preferably, the coverage determination unit comprises: The meta-path construction subunit is configured to construct three core meta-paths based on the fine-extracted heterogeneous entities and the mined multi-level relationships, and construct subgraphs based on each meta-path independently, and fuse subgraph embeddings through a multi-head attention mechanism. The dynamic adjustment subunit is configured to dynamically adjust based on the historical contribution of the attention weight to the coverage of the path, and construct a multi-dimensional knowledge graph.

[0015] Compared with the prior art, the beneficial effects of the present application are as follows: Through hierarchical collection and preprocessing of multi-source heterogeneous data, the data integrity and consistency are ensured, and reliable support is provided for performance evaluation, the dynamic indicator model based on machine learning realizes real-time adaptation of the indicator system and crawling and data, improves the evaluation accuracy, constructs the evaluation system, provides a scientific tool for supervision, and optimizes the efficiency of fund allocation.

[0016] Other features and advantages of the present application will be described in the following description, and some will become apparent from the description, or will be understood by those skilled in the art. The purpose and other advantages of the present application can be achieved and obtained by the structure specifically pointed out in the written description and the drawings.

[0017] The technical solutions of the present application will be further described in detail below with the help of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, for explaining the present application, and do not constitute a limitation on the present application. In the drawings: Figure 1 The structure diagram of the dynamic performance evaluation indicator optimization system based on machine learning in the embodiments of the present application. DETAILED DESCRIPTION

[0019] The preferred embodiments of the present application will be described below in conjunction with the drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and do not limit the present application.

[0020] The present application proposes a dynamic performance evaluation indicator optimization system based on machine learning, as shown in Figure 1, comprising: The data acquisition module is configured to acquire multi-source heterogeneous data in layers, and simultaneously, fine process the crawled files; A preprocessing module is configured to perform fusion preprocessing on the multi-source heterogeneous data and the refined processing file, and construct a mapping relationship set of crawling semantic labels and business dimensions; A model construction module is configured to learn the mapping relationship set based on a machine learning technique, and construct and train a dynamic indicator model; A thread optimization module is configured to perform verification analysis on the dynamic indicator model, determine an optimization thread scheme for each dynamic indicator, and perform adaptive optimization on the dynamic indicator model.

[0021] In this embodiment, the hierarchical acquisition is classified collection according to the level or type of data sources, forming a multi-dimensional data network, such as an enterprise level: collecting the vocational skills training fund management method and financial allocation records published on the official website of an enterprise; an organization level: collecting device usage logs and teacher allocation tables of a training organization; and a student level: collecting student employment contracts and vocational qualification certificate acquisition records stored in a blockchain.

[0022] The multi-source heterogeneous data is data of different sources and different structures, including structured, semi-structured and unstructured data.

[0023] The structured data includes financial allocation amount (numeric type) and training period (time type). The unstructured data includes crawled file text and student interview audio. The semi-structured data includes XML format operation reports submitted by training organizations.

[0024] In this embodiment, the crawled files are regulations, notices, management methods and the like related to vocational skills training published by governments at all levels or enterprises.

[0025] In this embodiment, the refinement processing is format cleaning, semantic analysis and targeted identification of the crawled files, and extraction of core information, such as removing HTML tags in the crawled files and retaining pure text; identifying core terms such as “fund audit” and “training qualification” in the crawled files; Determining the target objects of crawling (such as enterprise regulatory departments and training organizations).

[0026] In this embodiment, the fusion preprocessing is to integrate data of different sources, clean abnormal values, and unify formats to prepare for modeling.

[0027] In this embodiment, the crawling semantic label is a core semantic keyword extracted from the crawled text, used to represent the crawling intention.

[0028] The business dimension is a specific business field involved in performance evaluation, used for classified management of data, such as a fund use dimension (allocation rate, audit compliance rate), a training quality dimension (student pass rate, teacher qualification rate), and an employment effect dimension (employment rate, salary growth rate).

[0029] The mapping relationship set is a corresponding relationship set of crawling semantic labels and business dimensions, and is used to guide index design, for example, a "funds supervision" label is mapped to a "funds use" dimension, and a "training qualification audit" label is mapped to a "training quality" dimension.

[0030] In this embodiment, the dynamic index model is a model that can adaptively adjust indexes and their weights according to crawling and data changes, and specifically is a three-layer architecture design: a feature extraction layer (CNN), a time series analysis layer (LSTM), and a weight evolution layer (genetic algorithm). The dynamic adjustment mechanism is to retrain the model according to crawling updates every quarter.

[0031] In this embodiment, the verification analysis is to evaluate the accuracy, crawling coverage, and risk resistance of the model.

[0032] In this embodiment, the optimization thread scheme is an optimization strategy formulated for each index, including algorithms, resources, time sequences, etc., for example, for the funds arrival rate index: using a genetic algorithm to optimize the weight, allocating 2 CPU cores, and updating once a week, and for the new crawling index AI training penetration rate: using meta-learning to quickly adapt, and preferentially allocating GPU resources.

[0033] In this embodiment, the adaptive optimization is that the model automatically adjusts the index system according to the verification results without human intervention, for example, when it is found that the "training satisfaction" index has decreased in explanatory power, the "course completion rate" index is automatically removed and added, and within 48 hours after crawling update, the model automatically updates the weights of related indexes.

[0034] The beneficial effects of the above technical solutions are: through hierarchical collection and preprocessing of multi-source heterogeneous data, the data integrity and consistency are ensured, reliable support is provided for performance evaluation, a dynamic index model based on machine learning is used to realize real-time adaptation of the index system to crawling and data, improve evaluation accuracy, build an evaluation system, provide a scientific tool for supervision, and optimize the efficiency of fund allocation.

[0035] The present application provides a kind of dynamic performance evaluation index optimization system based on machine learning, data acquisition module includes: The first acquisition unit is used to collect structured data in the field of vocational skills training. The second acquisition unit is used to collect operation data of training institutions using Internet of Things sensor network, wherein the operation data is related to equipment utilization rate, teacher class hour proportion and site idle rate. The third acquisition unit is used to grab employment data of students through a blockchain storage platform. The structured data, operation data and employment data constitute multi-source heterogeneous data.

[0036] Preferably, the data acquisition module further comprises: A fourth acquisition unit is configured to use a directional web crawler to crawl the crawling files of the publishing data source, wherein the crawler is configured with an anti-crawling strategy, an anti-crawling mechanism for the crawling file webpage, and data grabbing is completed through the simulation interaction of a headless browser; A file traversal unit is configured to traverse the mark tags in the crawling files, and replace the corresponding tags with corresponding characters according to the types of the mark tags, wherein the characters are non-encrypted characters and encrypted characters; A format processing unit is configured to scan the replaced text, combine the continuously appearing characters into encrypted fields, and perform position radiation on the encrypted fields and the remaining text in the crawling text except the mark tags, to obtain an encrypted processing file, wherein the processing file is the fine processing result.

[0037] In this embodiment, structured data is data with fixed format and clear logic, which can be directly stored in a database and retrieved by field. For example, financial grant data: grant amount, grant period (monthly / quarterly grant), fund use classification (equipment procurement / teacher training); Training project data: training category (such as "AI skill training" and "electrician vocational qualification training"), number of participants (such as 120 people in a certain period of electrician training), subsidy standard (1500 yuan per person).

[0038] In this embodiment, the Internet of Things sensor network is a network of embedded sensor devices that collect physical world data, enabling real-time monitoring. For example, device utilization rate: deploy RFID sensors in the training institution's practical workshop to monitor the on-time length of machines and computers (such as a certain machine with a daily average usage of 6.5 hours); teacher class hour ratio: record teacher teaching time through classroom attendance systems (RFID + camera) to calculate the class hour ratio of full-time teachers (such as a certain institution with a full-time teacher class hour ratio of 80% per week); site idle rate: install infrared sensors in theory classrooms to count the daily idle time (such as a certain classroom with a daily idle time of 3 hours).

[0039] The blockchain storage platform uses blockchain technology to hash and encrypt data for storage, ensuring that it cannot be tampered with and is traceable. For example, employment contract storage: the electronic contract signed by the student and the enterprise is stored through the blockchain smart contract, storing the contract hash value, signing time (such as May 10, 2025), and enterprise business information; vocational qualification certificate data: certificate information such as electrician certificates and AI skill certifications obtained by students, storing certificate numbers, issuing agencies, and acquisition times (such as June 15, 2025); salary stream data: through the integration of enterprise financial systems and blockchains, the first month's salary (such as 5000 yuan) and salary payment time of the student are encrypted and stored.

[0040] In this embodiment, the directional web crawler is a program for targeted data crawling of a specific website, which has anti-crawling ability, for example, crawling the "Crawling Regulations" column under the "2025 Vocational Skill Training Subsidy" PDF file and HTML text of the official website of a certain provincial department of human resources; crawling the "Notice and Announcement" of the official website of the department of human resources "Vocational Skill Training Fund Performance Evaluation Method" annex table, specifically: based on the Scrapy framework of Python to build a crawler, configure dynamic IP pool (≥20 IP polling) and User-Agent pool (simulate Chrome / Firefox browser, etc.); use Puppeteer headless browser to simulate manual clicking, scrolling and other operations to break through the anti-crawling mechanism of JS rendering (such as clicking the "More Crawling" button to load hidden content).

[0041] In this embodiment, the mark tag is a tag in an HTML / XML file used to define the format of the content (such as 、 ), such as paragraph label : Replacing crawling files Strengthening capital supervision For "strengthening financial supervision"; Title label <h2>: Replacing< / h2> <h2>Subsidy application process< / h2> For "subsidy application process".

[0042] In this embodiment, the encryption field and the position radiation are to merge continuous characters into an encrypted string, and to keep semantic association according to label position, such as the original text "application for subsidy needs to submit: 1. Training institution qualification certificate; 2. Student list" is processed into "[A1] application for subsidy needs to submit: [A2] 1. Training institution qualification certificate; [A3] 2. Student list ", wherein the [A1], [A2] and the like marks keep the original position relationship, specifically: regular expression matches continuous characters (such as \s+ matches multiple spaces), and is replaced by an encryption field; a position mapping table is generated to record the starting position of the encryption field in the original text, so as to facilitate subsequent semantic analysis (such as storing {"A1":{"start":0,"end":6},"A2":{"start":7,"end":12}} in JSON format).

[0043] In this embodiment, the vocational skill training includes government vocational training and enterprise vocational training.

[0044] The beneficial effects of the above technical solutions are: through the correlation analysis of structured data and institution operation data, the training institutions with low fund use efficiency are identified; based on the mapping of blockchain employment data and crawled files, the actual influence of training projects on employment rate is verified, and multi-dimensional and high-reliability data support is provided for skill training performance evaluation.

[0045] The application provides a dynamic performance evaluation index optimization system based on machine learning. The data cleaning unit is used for performing structured data cleaning on the multi-source heterogeneous data. The system construction unit is used for defining a crawling object set related to performance evaluation to construct a multi-level user portrait label system for each object, wherein the multi-level user portrait label system is composed of core labels, important labels and general labels. The score determination unit is used for extracting text information from the to-be-processed file, and counting the occurrence times of user portrait labels with a label grade greater than or equal to a preset grade for each object to obtain a label matching degree score. The target determination unit is used for determining an object with a label matching degree score greater than a preset score as a target crawling object of the to-be-processed file. The parsing unit is configured to perform hierarchical semantic parsing on the target crawling object according to the target crawling object, wherein the hierarchical semantic parsing is related to the regulatory class semantics of the regulatory part targeted by the enterprise and the operation class semantics of the training institution targeted by the crawling. The multi-label classification unit is configured to perform multi-label classification on the hierarchical semantic parsing result by using a bidirectional encoding representation model, output crawling semantic labels, and establish a mapping relationship set with the business dimensions of the cleaned structured data.

[0046] In this embodiment, the structured data cleaning is an operation of filtering abnormal values, processing missing values, and unifying formats on the structured data. For example, abnormal value processing: the "equipment utilization rate" record of a training institution is 120% (more than 100%), which is determined as an abnormal value and corrected; Missing value filling: when the "funding purpose" field in the financial allocation data is missing, the mode of the same type of training project is used to fill it in; Format unification: different date formats such as "2025 / 6 / 1" and "2025.06.01" are uniformly converted to "YYYY-MM-DD".

[0047] In this embodiment, the crawling object set is the subject related to the performance evaluation of vocational skill training, including various roles that affect or involve.

[0048] In this embodiment, the multi-level user portrait label system is a label set of different importance levels constructed for each crawling object, which is used to represent the business characteristics of the object. For example, the enterprise regulatory department: Core label (weight 0.6): funding performance audit, subsidy crawling formulation; Important label (weight 0.3): training institution qualification audit, employment data supervision; General label (weight 0.1): promotion standardization, training account inspection.

[0049] Training institution: Core label: equipment maintenance rate, teacher certificate rate; Important label: student employment rate, course completion rate; General label: site utilization rate, student satisfaction.

[0050] Specifically: based on crawling file high-frequency word statistics (such as extracting enterprise regulatory related keywords by using TF-IDF); expert scoring to determine label level (such as inviting 3 industry experts to rank the importance of labels); store the label system by using Excel or knowledge graph tools (such as Neo4j).

[0051] In this embodiment, the to-be-processed file is a crawling file text after format cleaning (remove HTML tags, merge consecutive characters, etc.).

[0052] In this embodiment, the label matching score is the number of times each crawling object in the crawled text appears, and the sum is weighted according to the label level, for example, "funds audit" (core label of enterprise supervision, weight 0.6) appears twice in the crawled text, and "qualification audit" (important label, weight 0.3) appears three times; The matching score = 2*0.6 + 3*0.3 = 1.2 + 0.9 = 2.1.

[0053] In this embodiment, the target crawling object is the crawling object whose label matching score exceeds the preset threshold, that is, the main body that mainly influences or regulates, for example, the preset threshold is 1.5, the enterprise supervision department has a matching score of 2.1>1.5, and is determined as the target object; the training institution has a matching score of 0.8<1.5, and is temporarily not included in the target range (if the crawling does not involve the operation of the institution).

[0054] In this embodiment, the hierarchical semantic analysis is to classify the semantic of the crawled text according to the target crawling object, and extract the core semantics in different dimensions.

[0055] In this embodiment, the bidirectional encoding representation model (BERT) is a pre-training language model based on Transformer, which is used for text semantic understanding and multi-label classification, for example, input the crawled paragraph "establish a training fund performance audit system and carry out it once a quarter", and the BERT outputs the labels "funds supervision" and "audit period"; for "training institutions need to be equipped with AI teaching equipment", the output labels are "equipment requirements" and "training content".

[0056] In this embodiment, the crawling semantic label is the crawling core semantic keyword output by the BERT model, which is used to represent the crawling intention, such as "funds supervision", "training qualification", and "employment tracking".

[0057] In this embodiment, the business dimension is the specific business field of performance evaluation, which is used for classified management of data and indicators, for example: the fund use dimension (corresponding to the "funds supervision" label); The training quality dimension (corresponding to the "teacher requirement" label); the employment effect dimension (corresponding to the "employment tracking" label).

[0058] The beneficial effects of the above technical solutions are: after the structured data cleaning, reliable data basis is provided for subsequent modeling, the target object recognition accuracy is improved through the multi-level label system, the index design deviates from the crawling core is avoided, the performance evaluation index and the crawling intention are deeply bound, the funds audit compliance rate index is designed based on the mapping of labels and dimensions, and the fit degree of the evaluation system and the crawling is improved.

[0059] The present application provides a dynamic performance evaluation index optimization system based on machine learning, and the model construction module comprises: The reinforcing processing unit is configured to perform reinforcing processing on feature weights of core semantics associated with the targeted crawling object; The weakening processing unit is configured to count a reinforcing weight amount of the targeted crawling object and perform weakening processing on feature weights of core semantics associated with non-targeted crawling objects according to the reinforcing weight amount; The model construction unit is configured to construct a dynamic index model based on the weakening processing result, the reinforcing processing result, and the mapping relationship set.

[0060] Preferably, the model construction unit comprises: The structure construction subunit is configured to construct a text-label-association three-layer coding structure for the targeted crawling core semantics after the reinforcing processing; The mechanism deployment subunit is configured to deploy a teacher-student feature distillation mechanism for the non-targeted semantics after the weakening processing; The constraint conversion subunit is configured to convert the crawling-business mapping relationship set into a graph structure constraint; The model subunit is configured to obtain a dynamic index model based on the graph structure constraint, the three-layer coding structure, and the graph structure constraint.

[0061] In this embodiment, the targeted crawling object is a crawling main influence subject (such as an enterprise regulatory department or a training institution) determined by the preprocessing module.

[0062] In this embodiment, the core semantics is a crawling core intent keyword associated with the targeted object, which plays a decisive role in the design of the performance evaluation index.

[0063] The feature weight reinforcing improves the feature importance of the core semantics in the model through an algorithm, so as to maximize the influence of the core semantics on the index design. For example, for the "funding audit cycle" semantics, the feature weight is increased from the default 0.5 to 0.8 using the attention mechanism; in the convolutional neural network (CNN), the feature map activation value containing the "audit" keyword is enhanced through weight matrix weighting.

[0064] The non-targeted crawling object is a crawling object (such as a "employment enterprise" that is not focused on) whose label matching degree does not reach a threshold. For example, a certain training subsidy crawling mainly regulates the behaviors of enterprises and institutions, and the "employment enterprise" is a non-targeted object.

[0065] The reinforcing weight amount statistics is to calculate the total reinforcing degree of the targeted semantics, which serves as a basis for weakening the non-targeted semantics.

[0066] The feature weight weakening is to proportionally reduce the feature influence of the non-targeted semantics according to the targeted reinforcing degree. For example, if the total reinforcing weight is 2.1, the non-targeted semantic feature weight is uniformly multiplied by a decay coefficient = 1 / (1+2.1)=0.32, and the "enterprise recruitment process" semantic weight is reduced from 0.4 to 0.128.

[0067] Text layer encoding is a semantic vectorization representation of crawled text, capturing global and local semantics, for example, input the crawled clause "quarterly fund audit", extract a 768-dimensional semantic vector with BERT, where the word vectors of words such as "quarterly" and "audit" are activated.

[0068] Label layer encoding is to build an association graph of core labels, representing the semantic relationship between labels, for example, the core label "fund audit" is connected to the "penalty threshold" through the "regulatory process" relationship, and the edge weight = co-occurrence frequency (such as appearing 3 times simultaneously in crawling, weight 0.6).

[0069] The association layer encoding is to calculate the semantic similarity between the label and the existing indicator, and generate an indicator enhancement vector, for example, the cosine similarity between the label "fund audit" and the indicator "audit compliance rate" is 0.85, and the association vector is [0.85, 0.15] (15% is the association degree of other indicators).

[0070] In this embodiment, the teacher network is a pre-trained high-capacity model used to extract non-targeted global semantic features, and the student network is a lightweight model that learns key information under the guidance of targeted features and suppresses redundant features. Feature distillation through knowledge transfer allows the student network to retain only features related to targeted semantics and weaken irrelevant features, for example, after distillation, the non-targeted semantic "enterprise recruitment process" only retains the "account information review" sub-feature (indirectly related to fund supervision), and other features (such as "interview process") are suppressed.

[0071] The crawling-business mapping relationship set is the corresponding relationship between the crawled semantic labels and the performance evaluation business dimensions (such as "fund audit" -> "fund use" dimension), for example, the mapping pairs are: ("teacher certification" label, "training quality" dimension), ("employment tracking" label, "employment effect" dimension).

[0072] In this embodiment, the graph structure constraint is to convert the mapping relationship into a graph structure, with nodes representing labels and dimensions and edges representing association relationships, used to constrain the training direction of the model, for example, the nodes "fund audit" and "fund use" are connected through an edge, and the edge weight = mapping confidence (such as 0.9), requiring the model to preferentially associate "fund audit" to the "fund use" dimension when designing indicators.

[0073] The beneficial effects of the above technical solutions are: the accuracy of capturing key information by strengthening the core semantic model is improved, the non-targeted semantic weakening enhances the model's anti-interference ability, and the influence of irrelevant features on indicator design is reduced. Through the three-layer encoding structure, the semantic similarity of label-indicator association is improved, and the graph structure constraint ensures that the indicator design conforms to the crawling business dimension mapping, avoiding deviation of the indicator system from the crawling intent.

[0074] The application provides a dynamic performance evaluation index optimization system based on machine learning. A computing unit is configured to calculate the prediction accuracy and recall rate of the evaluation index system for each core target in the dynamic index model by using hierarchical cross-validation, introduce uncertainty quantification, output the confidence interval of the prediction result, and identify the difference between cognitive uncertainty and accidental uncertainty of the dynamic index model; A coverage determination unit is configured to construct a multi-dimensional knowledge graph and calculate the coverage of the dynamic index model for the crawled clauses by using a graph convolution network. A gap positioning unit is configured to position the index gap by using a semantic difference algorithm for new published crawling, and mark the crawling mismatch point. An analog unit is configured to simulate extreme scenario impact, test the sensitivity of the dynamic index, calculate the coefficient of variation of the index in each window by using a sliding window verification, and identify the index drift node. A scheme acquisition unit is configured to obtain the optimization thread scheme of each dynamic index from a resource-strategy matching table of deep reinforcement learning based on the difference, coverage, crawling mismatch point and index drift node, wherein the optimization thread scheme is related to the execution algorithm, resource, priority and execution timing of the dynamic index.

[0075] Preferably, the nodes of the multi-dimensional knowledge graph are crawling clauses and evaluation indexes, and the edges are semantic association and constraint relationship.

[0076] Preferably, the dynamic index is a static weight type index, a dynamic feedback type index and a new scene index of the dynamic index model.

[0077] Preferably, the coverage determination unit comprises: An meta-path construction subunit is configured to construct three core meta-paths based on fine extraction of heterogeneous entities and mining of multi-level relationships, construct a subgraph based on each meta-path, and fuse subgraph embedding by using a multi-head attention mechanism. A dynamic adjustment subunit is configured to dynamically adjust based on the historical contribution of attention weight to coverage, and construct a multi-dimensional knowledge graph.

[0078] In this embodiment, the hierarchical cross-validation divides the data into a training set and a test set after layering by features, thereby avoiding the influence of data bias on evaluation. For example, layering by time: the training data of Hebei Province from 2024 to 2025 is layered by quarter, and each fold contains quarterly data. Layering by institution type: training institutions are divided into two layers of "public" and "private", and each layer contains data of two types of institutions.

[0079] Uncertainty quantification is to evaluate the uncertainty of model prediction, distinguish cognitive uncertainty (lack of knowledge) and accidental uncertainty (data noise), for example, cognitive uncertainty: new crawling "AI training" data is insufficient, the model's prediction confidence interval of "AI certification rate" index is wide (such as [60%, 85%]); accidental uncertainty: student satisfaction data is affected by temporary factors, with a large fluctuation and a confidence interval of [75%, 80%].

[0080] In this embodiment, the crawling coverage refers to the degree to which the index system covers the crawling clauses, and quantifies the degree of fit between the index and the crawling, for example, a crawling contains 10 clauses, and the index system covers 8 clauses, with a coverage of 80%; the core clause "annual audit" is covered by the index "annual audit completion rate", which contributes a higher weight.

[0081] In this embodiment, the crawling mismatch point refers to the clause that the new crawling requirement does not match the existing index system, and a new index needs to be added, for example, the crawling requirement "track the salary of students 3 months after employment", and the existing index only tracks "the first month's salary", which is marked as a mismatch point.

[0082] In this embodiment, the extreme scenario impact test is to simulate abnormal situations (such as sudden reduction of financial allocation) to test the sensitivity of the index, for example, simulate a 30% reduction in financial allocation, and observe the fluctuation range of the "funds in place rate" index (such as from 90% to 65%, with a sensitivity of (90-65) / 30=0.83).

[0083] In this embodiment, the sliding window verification is to use a fixed window to slide through the data and calculate the stability of the index, for example, the window length is 3 months, and the coefficient of variation (CV) of the "student pass rate" in each window is calculated, if the CV of a certain window is 25%>15%, it is determined as a drift node.

[0084] In this embodiment, the index drift node is the time point at which the statistical characteristics of the index change significantly, and the model needs to be adjusted, for example, in May 2025, the CV of the "AI training employment rate" index increased from 10% to 25%, due to the adjustment of training content caused by new crawling, and it is identified as a drift node.

[0085] In this embodiment, the resource-strategy matching table is a predefined combination of algorithms, resources, and priorities for DRL to call, for example, algorithm: genetic algorithm (GA), meta-learning (MAML); resources: CPU core number 1-16, GPU memory 0-4GB; priority: high (crawling mismatch point index), medium (drift node index), low (stable index).

[0086] In this embodiment, the optimization thread solution is an optimization strategy customized for each indicator, including algorithms, resources, timing, etc., such as: Indicator "Funds Audit Compliance Rate": Algorithm: Genetic Algorithm (100 iterations); Resources: CPU 12 cores, GPU 2GB; Priority: High (crawl core clauses); Timing: Execute every Monday at 2am.

[0087] In this embodiment, heterogeneous entity fine extraction is to extract multiple types of entities (crawl, indicator, subject, etc.) from crawled text, such as: Entity Type: Crawl Clause "AI Training Subsidy in 2025", Indicator "AI Training Participation Rate", Subject "Provincial Department of Human Resources".

[0088] In this embodiment, multi-level relationship mining is to identify different types of relationships between entities (constraints, time, hierarchy, etc.), such as: Constraint Relationship: Crawl → Mandatory → Indicator "AI Training Assessment"; Time Relationship: Crawl Effective Date → Prior to → Indicator Assessment Period.

[0089] In this embodiment, core meta-path construction is to define the key path of crawl-indicator association, which is used for graph convolution, such as: P-C-I path (Crawl → Constraint → Indicator): Crawl "AI Training Assessment" → Mandatory Constraint → Indicator "AI Assessment Pass Rate"; P-T-I path (Crawl → Time → Indicator): Crawl Effective Date 2025.1 → Indicator Assessment Period 2025Q1.

[0090] In this embodiment, dynamic attention weight adjustment is to adaptively adjust the weight according to the historical contribution of the meta-path to the coverage, such as: P-C-I path historical contribution degree is high (improve coverage by 15%), weight from 0.3 to 0.4; P-T-I path contribution is low, weight from 0.3 to 0.2.

[0091] The beneficial effects of the above technical solutions are: Based on hierarchical cross-validation to improve prediction accuracy, based on uncertainty quantification to help identify data insufficient scenarios, guide data collection priority, based on multi-dimensional knowledge graph to reduce crawl coverage calculation error, based on semantic difference algorithm to realize indicator gap positioning within 48 hours after new crawl is published, based on extreme scenario test to identify high-risk indicators, adjust weight in advance, based on deep reinforcement learning driven thread solution to improve resource utilization, based on meta-path dynamic adjustment to improve knowledge graph coverage calculation efficiency.

[0092] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A dynamic performance evaluation index optimization system based on machine learning, characterized in that, include: The data acquisition module is used to acquire multi-source heterogeneous data in a layered manner, and at the same time, to perform fine-grained processing on the crawled files; The preprocessing module is used to perform fusion preprocessing on the multi-source heterogeneous data and refined processing files to construct a mapping relationship set between crawled semantic tags and business dimensions; The model building module is used to learn the mapping relationship set based on machine learning technology, and to build and train a dynamic index model. The thread optimization module is used to verify and analyze the dynamic indicator model, determine the optimization thread scheme for each dynamic indicator, and perform adaptive optimization on the dynamic indicator model.

2. The dynamic performance evaluation index optimization system based on machine learning according to claim 1, characterized in that, The data acquisition module includes: The first data collection unit is used to collect structured data in the field of vocational skills training. The second data acquisition unit is used to collect operational data of the training institution using an Internet of Things sensor network. The operational data is related to equipment utilization rate, teacher class hours ratio, and venue vacancy rate. The third data collection unit is used to capture students' employment data through a blockchain-based evidence storage platform. The structured data, operational data, and employment data constitute multi-source heterogeneous data.

3. The dynamic performance evaluation index optimization system based on machine learning according to claim 1, characterized in that, The data acquisition module further includes: The fourth collection unit is used to crawl files from the published data source using a targeted web crawler. The crawler is configured with anti-crawling strategies to target the anti-crawling mechanisms of the web pages containing the crawled files. Data is crawled by simulating interaction through a headless browser. The file traversal unit is used to traverse the tags in the crawled file and replace the corresponding tags with corresponding characters according to the type of the tags, wherein the characters are unencrypted characters and encrypted characters; The format processing unit is used to scan the replaced text, merge consecutively occurring characters into an encrypted field, and use the encrypted field to perform position radiation on the remaining text in the crawled text excluding the tags, to obtain an encrypted file to be processed, wherein the file to be processed is the result of fine-tuning.

4. The dynamic performance evaluation index optimization system based on machine learning according to claim 3, characterized in that, The preprocessing module includes: A data cleaning unit is used to perform structured data cleaning on the multi-source heterogeneous data; The system construction unit is used to define the set of crawled objects associated with performance evaluation and to build a multi-level user profile tag system for each object. The multi-level user profile tag system consists of core tags, important tags and general tags. The score determination unit is used to extract text information from the file to be processed and count the number of times user profile tags with a tag level greater than or equal to a preset level appear for each object, so as to obtain a tag matching score. The target determination unit is used to determine objects whose tag matching score is greater than a preset score as the target crawling objects of the file to be processed. The parsing unit is used to perform hierarchical semantic parsing on the file to be processed according to the target crawling object, wherein the hierarchical semantic parsing is related to the regulatory semantics of the enterprise supervision part and the operational semantics of the training institution. The multi-label classification unit is used to perform multi-label classification on the hierarchical semantic parsing results using a bidirectional encoding representation model, output the crawled semantic labels, and establish a mapping relationship set with the business dimensions of the cleaned structured data.

5. The dynamic performance evaluation index optimization system based on machine learning according to claim 4, characterized in that, The model building module includes: The enhancement processing unit is used to enhance the feature weights of the core semantics associated with the targeted crawled object; The weakening processing unit is used to calculate the reinforcement weight of the targeted crawling object and weaken the feature weight of the core semantic associated with the non-targeted crawling object based on the reinforcement weight. The model building unit is used to build and train a dynamic index model based on the weakening processing results, the strengthening processing results, and the mapping relationship set.

6. The dynamic performance evaluation index optimization system based on machine learning according to claim 5, characterized in that, The model building unit includes: The structural building sub-unit is used to construct a three-layer encoding structure of text-tag-association for the enhanced targeted crawling core semantics; The mechanism deployment subunit is used to deploy a teacher-student feature distillation mechanism for weakened non-targeted semantics; The constraint transformation subunit is used to transform the crawling-business mapping relationship set into graph structure constraints; The model subunit is used to obtain a dynamic index model based on the graph structure constraints, the three-layer coding structure, and the graph structure constraints.

7. The dynamic performance evaluation index optimization system based on machine learning according to claim 1, characterized in that, The thread optimization module includes: The calculation unit is used to calculate the prediction accuracy and recall of the evaluation index system for each core objective in the dynamic index model using hierarchical cross-validation, and introduce uncertainty quantification to output the confidence interval of the prediction results and identify the difference between cognitive uncertainty and accidental uncertainty in the dynamic index model. A coverage determination unit is used to construct a multidimensional knowledge graph and calculate the coverage of the crawling terms by the dynamic index model through a graph convolutional network. The gap location unit is used to locate the index gaps in newly published crawled data using a semantic difference algorithm and mark them as crawling mismatch points. The simulation unit is used to simulate extreme scenario impacts, test the sensitivity of the dynamic indicators, and use sliding window verification to calculate the coefficient of variation of the indicators within each window and identify indicator drift nodes. The scheme acquisition unit is used to obtain an optimized thread scheme for each dynamic indicator from the resource-policy matching table of deep reinforcement learning based on the differences, coverage, crawling mismatch points and indicator drift nodes. The optimized thread scheme is related to the execution algorithm, resources, priority and execution sequence of the dynamic indicator.

8. The dynamic performance evaluation index optimization system based on machine learning according to claim 7, characterized in that, The nodes of the multidimensional knowledge graph are crawling terms and evaluation indicators, and the edges are semantic associations and constraints.

9. The machine learning-based dynamic performance evaluation index optimization system according to claim 7, characterized in that, The dynamic indicators are static weighted indicators, dynamic feedback indicators, and new scenario indicators of the dynamic indicator model.

10. The dynamic performance evaluation index optimization system based on machine learning according to claim 7, characterized in that, The coverage determination unit includes: The meta-path construction subunit is used to construct three core meta-paths based on finely extracted heterogeneous entities and mined multi-level relationships, and to independently construct a subgraph based on each meta-path, and to fuse the subgraph embedding through a multi-head attention mechanism. The sub-units are dynamically adjusted based on the historical contribution of the path to the coverage, using attention weights, to construct a multi-dimensional knowledge graph.

Images