Professional ability label generation method and system based on multi-modal data fusion

Abstract

The invention discloses a professional ability label generation method and system based on multi-modal data fusion. The method comprises the following steps: acquiring multi-modal evaluation data of a professional; carrying out preprocessing on the multi-modal evaluation data; based on a professional domain knowledge graph, performing feature extraction on the preprocessed multi-modal evaluation data by adopting a multi-level semantic understanding architecture to generate feature vectors related to professional ability; fusing the feature vectors by adopting a hierarchical fusion strategy to obtain a fusion result; and on the basis of a fusion result, through collaborative decision-making of a rule engine, a machine learning model and a deep learning model, generating a personnel ability label in a professional label system covering multiple ability dimensions. Through multi-modal data fusion and a hierarchical collaborative decision-making mechanism, in combination with collaborative decision-making, the problems of insufficient label specialty and low accuracy caused by data splitting are effectively solved, and the accuracy, comprehensiveness and dynamic adaptability of professional ability label generation are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence decision-making technology, and in particular to a method and system for generating professional competency labels based on multimodal data fusion. Background Technology

[0002] Currently, in the field of professional competency assessment, existing technologies typically rely on structured assessment data or human experience judgment, making it difficult to effectively handle unstructured evaluation texts containing a large amount of subjective descriptions, such as annual assessment reports and democratic evaluation records. This results in a lack of professionalism, accuracy, and dynamic adaptability in the generated competency labels. At the same time, due to the failure to deeply integrate structured and unstructured multimodal data, existing methods cannot take into account both the objectivity of the data and the complexity of the semantics within a unified framework. This leads to the generalization of competency labels, one-sided profiles, and delayed updates, making it difficult to support scientific job matching and decision-making. Summary of the Invention

[0003] In view of this, the present invention proposes a method and system for generating professional competency labels based on multimodal data fusion, which can improve the accuracy of professional competency label generation. The present invention provides the following technical solution: A method for generating professional competency tags based on multimodal data fusion, the method comprising: Acquire multimodal evaluation data from professionals, including structured data and unstructured text data; The multimodal evaluation data is preprocessed, including cleaning and standardizing structured data, and performing professional word segmentation and noise reduction on unstructured text data. Based on a professional domain knowledge graph, a multi-level semantic understanding architecture is used to extract features from preprocessed multimodal evaluation data and generate feature vectors related to professional capabilities. The feature vectors are fused using a hierarchical fusion strategy, which includes fusion at the feature layer, decision layer, and result layer to obtain the fusion result. Based on the fusion results, personnel competency tags are generated in a professional tagging system covering multiple competency dimensions through collaborative decision-making by a rule engine, machine learning model, and deep learning model.

[0004] Optionally, the unstructured text data includes subjective evaluation descriptions, and the subjective evaluation descriptions account for more than 60% of the unstructured text data; The unstructured text data is derived from at least one of the following: annual performance evaluation report, democratic appraisal record, and special task appraisal. The structured data includes at least one of the following: professional competence assessment scores, behavioral record data, and qualification certification information.

[0005] Optionally, the cleaning and standardization of structured data includes: using an outlier detection algorithm to identify and process outlier data, and filling missing values ​​with a weighted strategy of industry benchmark values ​​and personal historical data; The professional word segmentation and noise reduction processing of unstructured text data includes: building a customized word segmenter based on a professional domain dictionary, extracting key time, numerical and event information through regular expressions, and using a natural language processing model to remove redundant modifiers and irrelevant expressions from the text.

[0006] Optionally, the multi-level semantic understanding architecture includes a three-layer processing structure of word level, sentence level, and document level; The word-level feature extraction employs a word vector training method enhanced by a professional dictionary to extract lexical semantic features; The sentence-level feature extraction employs a bidirectional long short-term memory network combined with an attention mechanism to extract sentence semantic features. The document-level feature extraction uses a Transformer encoder to extract document-level semantic features, and maps the document semantics to entities in the professional domain knowledge graph through entity association to achieve semantic enhancement. The feature vectors include numerical feature vectors extracted from structured data, text semantic feature vectors generated by fusing word-level, sentence-level, and document-level feature extraction results, and entity embedding vectors generated based on professional domain knowledge graphs.

[0007] Optionally, the fusion at the feature layer includes: aligning and concatenating the feature vectors of different modalities, and dynamically allocating the fusion weights of each modal feature vector based on information entropy; The fusion at the decision layer includes: calculating the decision weight of the corresponding modality evaluation data in label prediction based on the quality, source credibility, and semantic confidence of the multimodal evaluation data, and weighting and integrating the prediction information corresponding to each modality to generate a label probability distribution; The fusion process at the result layer includes: applying professional consistency rules to constrain and optimize the label prediction distribution, eliminating semantic conflicts, and outputting a fusion result that conforms to the logic of the professional domain.

[0008] Optionally, the rule engine is built based on a professional domain knowledge graph and includes preset judgment rules for each capability dimension; The machine learning model includes a gradient boosting tree model or a support vector machine, used for label prediction of structured data; The deep learning model includes a Transformer-based semantic understanding network for processing complex semantic information in unstructured text; The collaborative decision-making process includes: independently predicting labels for the fusion results using the rule engine, machine learning model, and deep learning model respectively, obtaining three sets of label prediction results, and then weighting and fusing the three sets of label prediction results based on preset weights to generate the final personnel capability labels.

[0009] Optionally, the method further includes: The quality assessment of the personnel competency tags is based on the confidence, consistency, and historical stability indicators of the personnel competency tags. The labels are dynamically revised based on the results of the quality assessment, and a multi-timescale label evolution tracking mechanism is established for short-term, medium-term and long-term purposes to achieve continuous updating and optimization of capability labels.

[0010] This invention further discloses a professional competency label generation system based on multimodal data fusion, comprising: The data acquisition module is used to acquire multimodal evaluation data from professionals, including structured data and unstructured text data. The preprocessing module is used to preprocess the multimodal evaluation data, including cleaning and standardizing structured data, and performing professional word segmentation and noise reduction on unstructured text data. The feature extraction module is used to extract features from preprocessed multimodal evaluation data based on professional domain knowledge graphs and adopt a multi-level semantic understanding architecture to generate feature vectors related to professional capabilities. The hierarchical fusion module is used to fuse the feature vectors using a hierarchical fusion strategy, which includes fusion at the feature layer, decision layer, and result layer to obtain the fusion result. The tag generation module is used to generate personnel capability tags in a professional tag system covering multiple capability dimensions based on the fusion results and through collaborative decision-making by a rule engine, machine learning model, and deep learning model.

[0011] The present invention further discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0012] The present invention further discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method.

[0013] According to the technical solution of this invention, a method and system for generating professional competency tags based on multimodal data fusion effectively solves the problems of insufficient tag professionalism, low accuracy, and delayed updates caused by the high proportion of subjective evaluation text and the separation of structured and unstructured data in existing technologies. Specifically, this method is based on a professional domain knowledge graph and uses a multi-level semantic understanding architecture to perform deep feature extraction on preprocessed multimodal evaluation data. It achieves multi-source information complementarity enhancement through a layered fusion strategy that integrates the feature layer, decision layer, and result layer. Combined with a collaborative decision-making mechanism of rule engine, machine learning model, and deep learning model, it automatically generates high-confidence competency tags within a professional tag system covering multiple dimensions such as safety, skills, management, and health. This significantly improves the accuracy, professionalism, and dynamic adaptability of the tags, effectively supporting precise matching of personnel to positions and scientific management decisions. Attached Figure Description

[0014] For illustrative and not limiting purposes, the present invention will now be described in conjunction with embodiments and accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the method for generating professional competency tags based on multimodal data fusion in an embodiment of the present invention. Figure 2 This is a schematic diagram of the components of the professional competency tag generation system based on multimodal data fusion in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the electronic device in an embodiment of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0016] It should be noted that, where there is no conflict, the embodiments and features of the embodiments in this application can be combined with each other. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0017] refer to Figure 1 This embodiment discloses a method for generating professional competency tags based on multimodal data fusion, which includes the following steps: S100: Obtain multimodal evaluation data from professionals, including structured data and unstructured text data.

[0018] In this embodiment, acquiring multimodal evaluation data of professionals is the starting step in the competency label generation process. This multimodal evaluation data consists of both structured and unstructured text data, which differ in data source, format, and semantic characteristics, and are used to characterize the comprehensive competency status of professionals.

[0019] Structured data includes, but is not limited to, professional competency assessment scores, behavioral record data, and qualification certification information. Behavioral record data includes quantitative indicators such as task completion rate, response timeliness, and operational compliance, while qualification certification information includes license levels, training certificates, and skill certifications. Structured data exists in the form of database tables, log files, or API interfaces, and has clearly defined field definitions and numerical attributes.

[0020] Unstructured text data originates from formal personnel documents such as annual performance appraisal reports, democratic evaluation records, and special task assessments. Its content includes a large number of subjective descriptive statements written by the evaluators, such as: "This comrade demonstrated a high sense of responsibility in emergency response" and "Communication and coordination skills need improvement." Such subjective evaluation descriptions account for over 60% of unstructured text data and are characterized by semantic ambiguity, diverse expressions, and implicit emotional biases.

[0021] In practice, the structured and unstructured text data mentioned above are collected by connecting to the organization's internal human resources management system, performance appraisal platform, document archive, and real-time behavior collection interface. Each record is then appended with a timestamp, data source identifier, and evaluator identity information to ensure sufficient contextual basis for subsequent preprocessing and fusion stages. This lays a multi-source heterogeneous data foundation for building a high-precision, high-coverage professional competency labeling system.

[0022] S200: Preprocess the multimodal evaluation data, including cleaning and standardizing structured data, and performing professional word segmentation and noise reduction on unstructured text data.

[0023] After acquiring multimodal evaluation data, targeted preprocessing is performed to ensure the reliability of subsequent feature extraction and fusion. Specifically, for structured data, outlier detection algorithms are first used to identify and process values ​​that deviate from the normal range. In this embodiment, unreasonable data points in assessment scores or behavioral records are identified through statistical methods or domain rules and then corrected or removed. For missing fields, a weighted strategy of industry benchmark values ​​and personal historical data is used to fill them. The industry benchmark value is taken from the statistical mean of the corresponding indicators for personnel in the same position or professional category, and the personal historical data is taken from the moving average of the relevant indicators for the personnel in the historical assessment period. The two are weighted according to a preset ratio. In a typical embodiment, the weight of the industry benchmark value is set to 0.4, and the weight of the personal historical data is set to 0.6, thereby generating reasonable replacements for missing values ​​and completing the cleaning and standardization of structured data. For unstructured text data, a customized word segmenter is built based on a professional domain dictionary. This dictionary, for example, covers keywords in dimensions such as safety, skills, and management, including terms like "QAR exceeding limits," "SOP compliance," and "emergency decision-making ability." The jieba Chinese word segmentation framework is used to load this custom dictionary to ensure the completeness and accuracy of the professional terminology. Subsequently, key time, numerical, and event information implicit in the text is extracted using regular expression templates and annotated in a structured manner. Simultaneously, a pre-trained language model is used to identify and remove redundant modifiers, colloquial expressions, and transactional content irrelevant to competency evaluation, retaining semantic fragments with clear evaluation orientation. After the above processing, the structured data is transformed into a standardized set of numerical values ​​with uniform format, no anomalies, and no missing data. The unstructured text is transformed into cleaned text with semantic focus, noise suppression, and preservation of professional expression characteristics. Both together form the high-quality input foundation for the subsequent feature extraction module.

[0024] S300: Based on a professional domain knowledge graph, it uses a multi-level semantic understanding architecture to extract features from preprocessed multimodal evaluation data and generate feature vectors related to professional capabilities.

[0025] After preprocessing the multimodal evaluation data, a multi-level semantic understanding architecture is used to extract features from the preprocessed data based on the professional domain knowledge graph, in order to generate feature vectors closely related to professional capabilities.

[0026] In this embodiment, the domain-specific knowledge graph is constructed using the Neo4j graph database. Taking the civil aviation patent field as an example, this embodiment includes five core entities: cadres, positions, skills, safety incidents, and training programs, as well as multiple predefined relationships, including "possessing skills," "holding a position in," "participating in training," and "involved in incidents." A structured knowledge base is formed through manually annotated triples, such as "<cadre A, possessing skills, flying in complex weather>" and "<cadre B, involved in an incident, a runway incursion incident in 2022>." This knowledge graph is used to enhance the domain relevance of semantic understanding, and embedding vectors of corresponding entities are generated through graph embedding methods as semantic priors for feature extraction.

[0027] The multi-layered semantic understanding architecture comprises three processing layers: word-level, sentence-level, and document-level. Specifically, at the word-level, the Word2Vec method is used to pre-train word vectors on corpora containing specialized terminology. These vectors are then combined with entity names from the aforementioned knowledge graph, such as "complex weather flight," "CRM training," and "runway safety," to construct a domain dictionary, enhancing the word vectors' ability to represent specialized vocabulary and outputting 256-dimensional semantic features. At the sentence level, evaluation sentences from the cleaned structured data in step S100 are input into a bidirectional long short-term memory network (BiLSTM) and combined with an attention mechanism to focus on keyword expressions related to the ability dimension, such as "accurate judgment" and "response delay." The algorithm generates 512-dimensional sentence semantic features based on the concepts of "organized and orderly" and "well-organized". At the document level, the entire evaluation text, such as an annual assessment report or a democratic appraisal record, is input into a Transformer-based encoder to extract global semantic representations. Through entity recognition and linking technology, information such as cadres, skills, events, and training mentioned in the document is mapped to corresponding nodes in the knowledge graph. These nodes are graph entity nodes in the knowledge graph that semantically match the named entities identified in the text. Entity linking technology is used to align text mentions with graph nodes. The embedding vectors of relevant entities in the graph are injected into the document representation to achieve semantic enhancement, ultimately outputting 768-dimensional document-level semantic features.

[0028] The semantic features from the three levels mentioned above are fused to form a text semantic feature vector. Simultaneously, the structured data, after standardization, constitutes a numerical feature vector, while the embedding vectors corresponding to the positions, skills, training, and events associated with the cadre in the knowledge graph constitute an entity embedding vector. Ultimately, three types of feature vectors are generated: a numerical feature vector extracted from the structured data, a text semantic feature vector generated by fusing word-level, sentence-level, and document-level feature extraction results, and an entity embedding vector generated based on a professional domain knowledge graph. These three vectors serve as inputs for subsequent hierarchical fusion modules.

[0029] S400: The feature vectors are fused using a hierarchical fusion strategy, which includes fusion at the feature layer, decision layer, and result layer to obtain the fusion result.

[0030] Furthermore, a hierarchical fusion strategy is adopted to fuse the feature vectors. The hierarchical fusion strategy includes fusion at the feature layer, decision layer and result layer to obtain the fusion result.

[0031] In feature layer fusion, the original feature values ​​of different modalities, such as structured data, unstructured text semantic scores, and real-time state stability rates, are first standardized: For positive characteristics (such as safety compliance rate and assessment results), the formula is used: Normalization is performed, and for negative features (such as the number of accidents or violation records), the following formula is used: To ensure semantic consistency, among which For the first The sample of professionals in the first The raw data for each modality, Let be the set of original feature values ​​of all samples in the j-th mode. The minimum value in the set. The maximum value in the set. These are the standardized feature values. Further, the probability distribution of each mode is calculated: ,in is the smoothing coefficient, m is the number of samples, and the modal entropy value is calculated based on this: To quantify the information content of each modality, and then through the difference coefficient The contribution of response modes to capability differentiation is ultimately used to assign feature layer fusion weights. Where n is the total number of modalities, dynamic weighted fusion based on information entropy is achieved. Simultaneously, conflict detection is performed using cosine similarity. When the similarity between feature vectors of different modalities is less than 0.3, a conflict is identified. A strategy prioritizing objective data combined with historical verification correction is adopted to resolve conflicts. For example, when flight QAR data is inconsistent with the text description, the consistency results of the person's three most recent similar scenario data are referenced for correction.

[0032] In the decision-making level fusion, the system constructs a three-dimensional confidence assessment model, integrating text quality score (based on grammatical correctness and information completeness, with a maximum score of 10), evaluator authority score (expert evaluation 10 points, peer 7 points, subordinate 5 points), and historical accuracy score (match rate between the last 10 evaluations and the actual results). A logistic regression model outputs a confidence score within the range of 0–1, and the prediction results of each modality are weighted and integrated according to confidence level. When decision conflicts arise, Bayesian inference is used for intelligent resolution, and its calculation formula is as follows: ,in, This represents the posterior probability in a Bayesian model. and These represent the two hypotheses: "objective data is reliable" and "subjective evaluation is reliable," respectively, with prior probabilities... Based on statistics from multiple historical conflict cases, the likelihood function is derived. By maximizing likelihood estimation optimization, we ensure that the conflict resolution accuracy reaches a preset threshold.

[0033] In the result layer fusion, the fused label prediction results are input into a consistency check library containing multiple rules. Logical conflicts are corrected through a dual mechanism of rule matching and semantic verification, and the final fusion result is output. This result serves as a reliable input for subsequent collaborative decision-making to generate personnel capability labels.

[0034] S500: Based on the fusion results, personnel capability tags are generated in a professional tagging system covering multiple capability dimensions through collaborative decision-making by a rule engine, machine learning model, and deep learning model.

[0035] After obtaining the fusion results through step S400, personnel capability tags are generated within a professional tagging system covering multiple capability dimensions through collaborative decision-making by a rule engine, machine learning model, and deep learning model. The professional tagging system is constructed based on civil aviation industry standards and includes four dimensions: safety capability (28 tags), professional skills (35 tags), management capability (22 tags), and health status (12 tags), totaling 97 core tags. Each tag has a unique code; for example, "Safety-001" indicates "strong safety awareness," and it is associated with over 1500 mapping rules between capability standards and tags. In the collaborative decision-making process, the rule engine is implemented based on the Drools platform, loading 1567 civil aviation professional rules, including 423 safety rules, 512 skill rules, 382 management rules, and 250 health rules. It performs logical matching on structured indicators in the fusion results, such as continuous flight exceeding 8 hours and lack of qualifications in complex weather, directly generating high-confidence labels. The machine learning model uses the LightGBM algorithm, pre-trained on a general personnel evaluation dataset and fine-tuned using 30,000 pieces of data specific to civil aviation cadres. For example, a learning rate of 0.01, a tree depth of 8, and 1000 iterations are set to classify and predict the fused feature vectors, outputting the probability distribution of each label. The deep learning model, for example, is based on the Qianwen large model, fine-tuned and adapted to the civil aviation field using LoRA. It utilizes over 100,000 annotated texts to deeply understand the semantic context in the fusion results, identifying implicit ability tendencies such as "decisive emergency response" and "insufficient teamwork," and generating corresponding label probability distributions. After the rule engine, machine learning model, and deep learning model run independently, a weighted voting method is used to fuse the prediction results of the three sets of labels. The weight of the rule engine is set to 0.35, the weight of the machine learning model is 0.35, and the weight of the deep learning model is 0.3. Labels whose weighted scores exceed the dynamic threshold are adopted as preliminary capability labels. Subsequently, the preliminary labels undergo multi-dimensional verification by the label quality assessment module. Labels that are conflicting (such as generating both "safety pacesetter" and "safety accident responsible person" at the same time) or have low confidence (such as labels generated based on the vague expression "needs improvement" with a confidence of ≤0.6) are corrected or filtered. The multi-dimensional verification includes four dimensions: accuracy, consistency, stability, and interpretability. Finally, the output is a personnel capability label that conforms to professional logic, is highly reliable, and interpretable, fully supporting subsequent personnel-job matching and management decisions.

[0036] S600: Perform a quality assessment on the personnel competency tags, which is based on the confidence level, consistency, and historical stability indicators of the personnel competency tags; dynamically correct the tags according to the results of the quality assessment, and establish a multi-timescale tag evolution tracking mechanism for short-term, medium-term, and long-term periods to achieve continuous updating and optimization of competency tags.

[0037] Specifically, after generating personnel competency tags, a quality assessment is performed on these tags. This quality assessment is based on three indicators: confidence level, consistency, and historical stability. Confidence level reflects the reliability of the tag generation process and is obtained by normalizing the probability score output from the collaborative decision-making stage. Its value range is 0–1. If the tag originates from a vague expression, such as "needs improvement" or "basically qualified," the confidence level is limited to no more than 0.6. Consistency refers to the logical rationality within the tag set. It is verified by matching against a consistency check library containing over 300 civil aviation professional rules. For example, the coexistence of "Safety-001 (strong safety awareness)" and "Safety-015 (fatigue flight risk)" within the same period is considered inconsistent. The consistency score is calculated based on the proportion of conflict-free tags. Historical stability measures the tag's ability to remain consistent over time. It is calculated by the frequency of the tag's occurrence over the past three assessment periods. For example, if "Professional Skills-012 (Advanced Flight Qualification Certification)" persists for 13 months, a stability score is generated using a sliding window averaging method. The three indicators are weighted equally (0.25 for each, with the remaining 0.25 for interpretability supplementary items) to form a comprehensive quality score. When the score is lower than the dynamic threshold, the label dynamic correction mechanism is triggered: low-confidence labels are downweighted or removed, high-confidence items are retained for inconsistent label combinations based on Bayesian inference results, and a manual review process is initiated for highly volatile labels.

[0038] Simultaneously, a multi-timescale tag evolution tracking mechanism is established to achieve continuous updating and optimization of capability tags: In the short term (17 days), a sliding window averaging method is used to monitor tag activity, and sudden events, such as security incident reports, triggering tag mutations are captured in real time; in the medium term (14 weeks), an exponential smoothing method is used to smooth the tag retention rate, calculated using the following formula: ,in, Let be the smoothed label retention rate in week t. This represents the original retention rate for the current week. The smoothing coefficient is used to identify steady-state changes in capability trends. Over a long-term scale (13 months), linear regression combined with quadratic polynomial correction is used to fit the tag evolution trajectory, determining capability improvement, plateauing, or decline trends, and adjusting tag weights or triggering capability development recommendations accordingly. The generation, correction, and evolution of all tags are logged in their entirety (including data source ID, fusion weights, decision model output, and correction basis) to ensure traceability and interpretability, ultimately achieving dynamic, accurate, and sustainable optimization of the tag system.

[0039] In summary, this implementation first acquires structured and unstructured multimodal evaluation data containing a high proportion of subjective descriptions, and preprocesses it using word segmentation and denoising enhanced by a civil aviation professional dictionary, as well as missing value imputation using industry averages and individual historical weighted averages. Then, based on a knowledge graph constructed using Neo4j containing five types of entities—cadres, positions, skills, safety incidents, and training programs—feature vectors are extracted using a three-layer semantic understanding architecture at the word, sentence, and document levels. A layered fusion strategy involving feature, decision, and result layers is then used to obtain the fusion results. Subsequently, a three-way collaborative decision-making process involving a rule engine, the LightGBM model, and the Qianwen big data model generates 97 professional tags. Finally, dynamic tag optimization is achieved through a four-dimensional evaluation of confidence, consistency, and historical stability, along with short-term, medium-term, and long-term multi-timescale tracking mechanisms. This significantly improves the accuracy, professionalism (covering multiple capabilities of civil aviation), dynamic adaptability (supporting real-time updates), and interpretability (complete generation path) of tag generation, effectively solving the problems of tag generalization, data fragmentation, update lag, and insufficient risk identification in existing technologies.

[0040] refer to Figure 2 This embodiment further discloses a professional competency label generation system based on multimodal data fusion, including a data acquisition module 21, a preprocessing module 22, a feature extraction module 23, a hierarchical fusion module 24, a label generation module 25, and a quality assessment and optimization module 26, which are described in detail below: The data acquisition module 21 is used to acquire multimodal evaluation data from professionals, including structured data and unstructured text data. The preprocessing module 22 is used to preprocess the multimodal evaluation data, including cleaning and standardizing structured data, and performing professional word segmentation and denoising on unstructured text data. The cleaning and standardization of structured data includes: identifying and processing abnormal data using an outlier detection algorithm, and filling missing values ​​using a weighted strategy of industry benchmark values ​​and personal historical data. The professional word segmentation and denoising on unstructured text data includes: constructing a customized word segmenter based on a professional domain dictionary, extracting key time, numerical, and event information using regular expressions, and simultaneously using a natural language processing model to remove redundant modifiers and irrelevant expressions from the text.

[0041] The feature extraction module 23 is used to extract features from preprocessed multimodal evaluation data based on a professional domain knowledge graph and employing a multi-level semantic understanding architecture to generate feature vectors related to professional competence. The multi-level semantic understanding architecture includes a three-layer processing structure: word-level, sentence-level, and document-level. Word-level feature extraction uses a professional dictionary-enhanced word vector training method to extract lexical semantic features. Sentence-level feature extraction uses a bidirectional long short-term memory network combined with an attention mechanism to extract sentence semantic features. Document-level feature extraction uses a Transformer encoder to extract document-level semantic features and maps document semantics to entities in the professional domain knowledge graph through entity association, achieving semantic enhancement. The feature vectors include numerical feature vectors extracted from structured data, text semantic feature vectors generated by fusing word-level, sentence-level, and document-level feature extraction results, and entity embedding vectors generated based on the professional domain knowledge graph.

[0042] The hierarchical fusion module 24 is used to fuse the feature vectors using a hierarchical fusion strategy. This strategy includes layer-by-layer fusion at the feature layer, decision layer, and result layer to obtain a fusion result. The fusion at the feature layer includes aligning and concatenating the feature vectors from different modalities, and dynamically allocating fusion weights for each modality's feature vectors based on information entropy. The fusion at the decision layer includes calculating the decision weights of the corresponding modality's evaluation data in label prediction based on the quality, source credibility, and semantic confidence of the multimodal evaluation data, and weighted integrating the prediction information corresponding to each modality to generate a label probability distribution. The fusion at the result layer includes applying professional consistency rules to constrain and optimize the label prediction distribution, eliminating semantic conflicts, and outputting a fusion result that conforms to the logic of the professional domain.

[0043] The tag generation module 25 is used to generate personnel ability tags in a professional tag system covering multiple ability dimensions based on the fusion result through collaborative decision-making by a rule engine, a machine learning model, and a deep learning model. The rule engine is constructed based on a professional domain knowledge graph and includes preset judgment rules for each ability dimension. The machine learning model includes a gradient boosting tree model or a support vector machine for processing tag prediction of structured data. The deep learning model includes a Transformer-based semantic understanding network for processing complex semantic information in unstructured text. The collaborative decision-making includes: independently predicting tags on the fusion result using the rule engine, machine learning model, and deep learning model respectively to obtain three sets of tag prediction results, and then weighting and fusing the three sets of tag prediction results based on preset weights to generate the final personnel ability tags.

[0044] The quality assessment and optimization module 26 is used to conduct quality assessments on the personnel competency tags. The quality assessment is based on the confidence, consistency, and historical stability indicators of the personnel competency tags. The tags are dynamically corrected according to the results of the quality assessment. At the same time, a multi-timescale tag evolution tracking mechanism with short-term, medium-term, and long-term time scales is established to achieve continuous updating and optimization of competency tags.

[0045] Figure 3 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown, the electronic device 50 includes: a processor 501, a memory 502, and a bus 503; The processor 501 and the memory 502 communicate with each other via the bus 503; the processor 501 is used to call the program instructions in the memory 502 to execute the methods provided in the above-described embodiments.

[0046] This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions that cause a computer to execute the methods provided in the above-described embodiments.

[0047] Those skilled in the art will understand that all or part of the steps of the above-described method implementation can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above-described method implementation. The aforementioned storage medium includes various storage media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0048] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0049] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0050] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating professional competency tags based on multimodal data fusion, characterized in that, The method includes: Acquire multimodal evaluation data from professionals, including structured data and unstructured text data; The multimodal evaluation data is preprocessed, including cleaning and standardizing structured data, and performing professional word segmentation and noise reduction on unstructured text data. Based on a professional domain knowledge graph, a multi-level semantic understanding architecture is used to extract features from preprocessed multimodal evaluation data and generate feature vectors related to professional capabilities. The feature vectors are fused using a hierarchical fusion strategy, which includes fusion at the feature layer, decision layer, and result layer to obtain the fusion result. Based on the fusion results, personnel competency tags are generated in a professional tagging system covering multiple competency dimensions through collaborative decision-making by a rule engine, machine learning model, and deep learning model.

2. The method for generating professional competency tags based on multimodal data fusion according to claim 1, characterized in that, The unstructured text data contains subjective evaluation descriptions, and the subjective evaluation descriptions account for more than 60% of the unstructured text data; The unstructured text data is derived from at least one of the following: annual performance evaluation report, democratic appraisal record, and special task appraisal. The structured data includes at least one of the following: professional competence assessment scores, behavioral record data, and qualification certification information.

3. The method for generating professional competency tags based on multimodal data fusion according to claim 2, characterized in that, The cleaning and standardization of structured data includes: using outlier detection algorithms to identify and process outlier data, and filling missing values ​​with a weighted strategy of industry benchmark values ​​and personal historical data; The professional word segmentation and noise reduction processing of unstructured text data includes: building a customized word segmenter based on a professional domain dictionary, extracting key time, numerical and event information through regular expressions, and using a natural language processing model to remove redundant modifiers and irrelevant expressions from the text.

4. The method for generating professional competency tags based on multimodal data fusion according to claim 1, characterized in that, The multi-level semantic understanding architecture includes a three-layer processing structure at the word level, sentence level, and document level. The word-level feature extraction employs a word vector training method enhanced by a professional dictionary to extract lexical semantic features; The sentence-level feature extraction employs a bidirectional long short-term memory network combined with an attention mechanism to extract sentence semantic features. The document-level feature extraction uses a Transformer encoder to extract document-level semantic features, and maps the document semantics to entities in the professional domain knowledge graph through entity association to achieve semantic enhancement. The feature vectors include numerical feature vectors extracted from structured data, text semantic feature vectors generated by fusing word-level, sentence-level, and document-level feature extraction results, and entity embedding vectors generated based on professional domain knowledge graphs.

5. The method for generating professional competency tags based on multimodal data fusion according to claim 1, characterized in that, The fusion process at the feature layer includes: aligning and concatenating the feature vectors of different modalities, and dynamically allocating the fusion weights of each modal feature vector based on information entropy; The fusion at the decision layer includes: calculating the decision weight of the corresponding modality evaluation data in label prediction based on the quality, source credibility, and semantic confidence of the multimodal evaluation data, and weighting and integrating the prediction information corresponding to each modality to generate a label probability distribution; The fusion process at the result layer includes: applying professional consistency rules to constrain and optimize the label prediction distribution, eliminating semantic conflicts, and outputting a fusion result that conforms to the logic of the professional domain.

6. The method for generating professional competency tags based on multimodal data fusion according to claim 1, characterized in that, The rule engine is built on a professional domain knowledge graph and includes preset judgment rules for each capability dimension; The machine learning model includes a gradient boosting tree model or a support vector machine, used for label prediction of structured data; The deep learning model includes a Transformer-based semantic understanding network for processing complex semantic information in unstructured text; The collaborative decision-making process includes: independently predicting labels for the fusion results using the rule engine, machine learning model, and deep learning model respectively, obtaining three sets of label prediction results, and then weighting and fusing the three sets of label prediction results based on preset weights to generate the final personnel capability labels.

7. The method for generating professional competency labels based on multimodal data fusion according to any one of claims 1-6, characterized in that, The method further includes: The quality assessment of the personnel competency tags is based on the confidence, consistency, and historical stability indicators of the personnel competency tags. The labels are dynamically revised based on the results of the quality assessment, and a multi-timescale label evolution tracking mechanism is established for short-term, medium-term and long-term purposes to achieve continuous updating and optimization of capability labels.

8. A professional competency label generation system based on multimodal data fusion, characterized in that, include: The data acquisition module is used to acquire multimodal evaluation data from professionals, including structured data and unstructured text data. The preprocessing module is used to preprocess the multimodal evaluation data, including cleaning and standardizing structured data, and performing professional word segmentation and noise reduction on unstructured text data. The feature extraction module is used to extract features from preprocessed multimodal evaluation data based on professional domain knowledge graphs and adopt a multi-level semantic understanding architecture to generate feature vectors related to professional capabilities. The hierarchical fusion module is used to fuse the feature vectors using a hierarchical fusion strategy, which includes fusion at the feature layer, decision layer, and result layer to obtain the fusion result. The tag generation module is used to generate personnel capability tags in a professional tag system covering multiple capability dimensions based on the fusion results and through collaborative decision-making by a rule engine, machine learning model, and deep learning model.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method of any one of claims 1-7.

Images