An agent analysis and standardization processing method for power industry data

Abstract

The application discloses an agent analysis and standardization processing method for power industry data, comprising the following steps: S1, acquiring multi-source data and preprocessing to form a sequence of to-be-analyzed segments; S2, constructing a multi-agent collaborative analysis mechanism and configuring analysis focus parameters; S3, performing semantic matching on the sequence of to-be-analyzed segments and adjusting matching priority; S4, performing sequence difference analysis on standard expressions, calculating bit difference values and representing sorting correlation degrees, and screening to-be-analyzed segments with analysis differences; S5, constructing evidence distribution values of candidate standard expressions according to the sequence difference analysis results, and screening standardization mapping results by support degree evaluation; S6, reconstructing standard expressions to generate standardization data records, and incrementally correcting analysis strategies. The application can realize automatic analysis and standardization conversion of multi-source data in the power industry under the multi-agent collaboration, and improve the recognition accuracy of non-standard data expression.

Description

Technical Field

[0001] This invention relates to the field of intelligent agent parsing technology, and in particular to an intelligent agent parsing and standardization processing method for power industry data. Background Technology

[0002] As the power industry becomes increasingly information-driven and digitalized, various business systems continuously generate massive amounts of data during operation. This data is typically scattered in origin, inconsistent in structure and expression, and the same business meaning may be described differently in different systems or data files. To support data governance, cross-system sharing, intelligent analysis, and subsequent business collaboration, it is usually necessary to first parse, organize, and standardize the power industry data, transforming it into a relatively unified standard expression.

[0003] In existing technologies, power data standardization processing is mostly achieved through manual rule configuration, field mapping tables, keyword matching, template conversion, or general semantic matching models. These methods are applicable when the data format is stable, the expression is fixed, and the number of standard fields is limited. However, their processing capabilities often depend on pre-defined matching relationships. When the data to be processed contains omitting expressions, contextual relationships, synonyms, non-standard descriptions, or multiple candidate standard expressions, simply relying on fixed rules or single-path matching can easily lead to mismatches, missed matches, and unstable candidate results.

[0004] Furthermore, the standardization of power industry data is not simply a matter of text replacement or field merging. The parsing results typically require a comprehensive assessment considering contextual relationships, standard resource constraints, and historical processing feedback. Existing methods largely lack multi-faceted collaborative parsing mechanisms for the same content, making it difficult to effectively identify differences between various parsing results and to promptly detect ranking conflicts and abnormal deviations between candidate standard expressions. This results in some ambiguous data segments still requiring manual review.

[0005] Meanwhile, existing standardization methods often prioritize one-time output results, lacking the incremental correction capability to use feedback results to influence subsequent parsing strategies. When deviations occur in the standardization results, the system can usually only correct them by manually modifying rules or re-maintaining the mapping table. It is difficult to automatically adjust the matching bias and candidate selection strategy in subsequent parsing processes based on feedback, affecting the continuous adaptability of standardization processing in complex power data scenarios.

[0006] Therefore, how to provide a method for intelligent agent parsing and standardized processing of data in the power industry is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose an intelligent agent parsing and standardization processing method for power industry data. This invention fully utilizes multi-agent collaborative parsing, dual-tower semantic matching, gradient boosting tree sorting, and Dempster-Shafer evidence fusion technology. It describes in detail the processing of multi-source power industry data from fragmented parsing, candidate standard expression matching, parsing divergence identification to standardization mapping screening. It has the advantages of strong parsing adaptability, stable standardization results, and strong feedback correction capability.

[0008] A method for intelligent agent parsing and standardization of power industry data according to an embodiment of the present invention includes the following steps: S1. Acquire multi-source data from the power industry to be processed and preprocess it to form a sequence of segments to be parsed; S2. Construct a multi-agent collaborative parsing mechanism for standardized processing, and configure parsing focus parameters for multiple agents based on the degree of contextual association between each segment in the sequence of segments to be parsed; S3. Each agent uses a dual-tower semantic matching method to perform semantic matching between the sequence of segments to be parsed and the standard resource library of the power industry, based on the parsing focus parameters. The matching priority is adjusted by using a gradient boosting tree sorting model to form multiple candidate sequences. S4. Perform sequence difference analysis on the standard expression in multiple candidate sequences, calculate the position difference of the same standard expression in different candidate sequences, and use Kendall rank correlation coefficient to characterize the order correlation between different candidate sequences. Combine with local outlier factors to screen out segments with analytical discrepancies. S5. For segments to be analyzed that have discrepancies, construct evidence allocation values ​​for candidate standard expressions based on the sequence difference analysis results, and use evidence fusion methods to assess support and screen standardized mapping results. S6. Based on the standardized mapping results, the sequence of segments to be parsed is reconstructed using standard representations to generate standardized data records. Based on the feedback results, the parsing strategy is incrementally corrected using an online mirror descent method.

[0009] Optionally, the preprocessing specifically includes: The multi-source data to be processed in the power industry is subjected to structural boundary analysis to identify the boundary positions between data contents with different structural forms, and the multi-source data is split into initial content fragments according to the boundary positions, wherein: The multi-source data includes: data content to be standardized from different power business systems, different data interfaces, or different file structures; The initial content segments are semantically connected. Based on the semantic similarity, contextual co-occurrence relationship and semantic breakpoint position between adjacent initial content segments, the initial content segments are merged or re-segmented to obtain semantic segments. Redundant expressions are compressed and invalid perturbations are removed from semantic segments. Repetitive expressions and anomalous markers that do not participate in the normalized parsing are removed, while the original order relationship between semantic segments is preserved. Based on the contextual position of the semantic fragment and the degree of association between adjacent semantic fragments, a fragment index and context association identifier are configured for the semantic fragment, forming a sequence of fragments to be parsed.

[0010] Optionally, the multi-agent collaborative parsing mechanism is a mechanism that differentiates multiple agents based on parsing focus parameters and achieves collaborative parsing through shared fragment states, weight adjustment, and candidate difference feedback. This mechanism includes a context-weighted agent, a standard resource-weighted agent, a historical feedback-weighted agent, and a divergence re-judgment agent, wherein: The parsing focus parameters are used to adjust the parsing control quantities of multiple agents during the semantic parsing process, so that different agents form different candidate standard expression generation paths when processing the same segment to be parsed. The context weight agent is used to adjust the scope of context reference for the current segment in the semantic matching process according to the degree of contextual association between adjacent segments in the segment sequence to be parsed; The standard resource weighting agent is used to adjust the candidate retention threshold of the standard expression in the candidate sequence according to the matching distance between the fragment to be parsed and the power industry standard resource library; The historical feedback weighting agent is used to adjust the matching weight of candidate standard expressions corresponding to the same type of fragments according to the hit frequency of the confirmed standardized mapping records. The divergence rematch agent is used to adjust the rematch range of segments with analytic divergence based on the position difference between multiple candidate sequences and local outlier results.

[0011] Optionally, S3 specifically includes: S31. Based on the parsing focus parameters corresponding to each agent, the context range of the segments to be parsed in the sequence of segments to be parsed is extracted and weighted, and the segments to be parsed after weighting are semantically encoded to obtain the segment semantic vectors corresponding to each agent. S32. Using the dual-tower semantic matching method, the standard expression items in the power industry standard resource library are semantically encoded to obtain standard semantic vectors, and the matching distance between the fragment semantic vector and the standard semantic vector is measured. S33. Based on the matching distance, select the standard expression terms corresponding to each segment to be parsed from near to far to form an initial candidate set. In the initial candidate set, extract the matching distance, contextual relevance, and parsing focus parameters corresponding to each standard expression term as ranking features. S34. Using a gradient boosting tree sorting model, the matching priority of the standard expression items in the initial candidate set is adjusted according to the sorting features, and the standard expression items are arranged according to the adjusted matching priority to form candidate sequences corresponding to multiple agents.

[0012] Optionally, the dual-tower semantic matching method specifically includes: Using the segments to be parsed in the sequence as segment-side matching objects and the standard expressions in the power industry standard resource library as standard-side matching objects, semantic vector transformation is performed using both the segment coding tower and the standard coding tower to obtain segment semantic vectors and standard semantic vectors, where: The fragment coding tower and the standard coding tower adopt a semantic coding structure with independent parameters and consistent output dimensions. They perform semantic coding on the fragment-side matching objects and the standard-side matching objects respectively, forming fragment semantic vectors and standard semantic vectors with dimensions ranging from 128 to 768. The standard coding tower generates standard semantic vectors based on standard expression items in the power industry standard resource library, and maps fragment semantic vectors and standard semantic vectors to the same semantic metric space; Based on the degree of contextual relevance of the segment to be parsed and the parameters of parsing focus, the vector weights in the semantic vector of the segment that are related to contextual relevance and standardization tendency are adjusted to form segment matching vectors corresponding to different agents; The cosine distance between the segment matching vector and the standard semantic vector is measured, and the standard expression terms are arranged in ascending order of cosine distance. The standard expression terms that are ranked first are selected as candidate standard expression terms. The number of candidates, K, is determined based on the degree of contextual relevance of the segment to be parsed, where: When the degree of contextual relevance is higher than the preset relevance threshold, K is set to 5 to 10; When the degree of contextual relevance is not higher than the preset relevance threshold, K is between 10 and 20. The candidate criteria expressions are truncated according to the number of candidates K to form an initial candidate set.

[0013] Optionally, S34 specifically includes: S341. Form candidate matching pairs by combining the fragment to be parsed with the standard expression terms in the initial candidate set, and construct the ranking feature vector of the candidate matching pairs based on the cosine distance, contextual relevance, and parsing focus parameters obtained from the dual-tower semantic matching. S342. A gradient boosting tree ranking model is adopted to perform round-by-round ranking fitting on the candidate matching pairs. In each round of ranking fitting, a regression tree is constructed based on the ranking residual of the candidate matching pairs in the previous round, and the ranking increment of the candidate matching pairs is output by the regression tree. S343. Accumulate the ranking increments obtained from multiple rounds of ranking fitting to form the ranking score of the candidate matching pair, and adjust the matching priority of the corresponding standard expression item in the initial candidate set according to the ranking score. S344. According to the adjusted matching priority, the standard expression items in the initial candidate set are serialized to form candidate sequences corresponding to multiple agents.

[0014] Optionally, S4 specifically includes: S41. For multiple candidate sequences corresponding to the same segment to be parsed, extract the standard expression and its candidate position in each candidate sequence, and use the standard expression that appears in multiple candidate sequences as the rank alignment object to construct the candidate rank table corresponding to the segment to be parsed. S42. Based on the candidate rank table, perform difference operations on the candidate positions of the same standard expression in different candidate sequences to obtain the position difference of the standard expression, and arrange the position difference of each standard expression in a matrix according to the source of the candidate sequence to form a candidate rank difference matrix. S43. Based on the candidate rank difference matrix, any two candidate sequences are used to form a sequence comparison pair, and the Kendall rank correlation coefficient is used to measure the rank correlation of the sequence comparison pair to form the sequence correlation matrix of the corresponding segment to be parsed. S44. Based on the candidate rank difference matrix and sequence correlation matrix, construct the divergence vector of the segment to be parsed, and use the local outlier factor method to determine the neighborhood density of the divergence vectors of multiple segments to be parsed. The segments to be parsed whose neighborhood density is lower than that of the surrounding segments to be parsed are identified as segments to be parsed with divergence.

[0015] Optionally, S44 specifically includes: S441. Based on the candidate rank difference matrix, the distribution of the position difference of each standard expression in the same segment to be parsed is characterized, the maximum position difference, the median of the position difference, and the proportion of non-zero position differences are determined. Based on the sequence correlation matrix, the lowest rank correlation coefficient is determined, which constitutes the segment divergence coordinates of the segment to be parsed. S442. Using the segment divergence coordinates as the discrimination object, the k-nearest neighbor reachability distance method is used to determine the local neighborhood of each segment to be parsed, and the local reachability density is formed based on the reachability distance between the segment to be parsed and its local neighborhood. S443. Based on the difference in local reachability density between the current segment to be parsed and neighboring segments, generate the local outlier factor of the corresponding segment divergence coordinates, and mark the corresponding segment to be parsed as a divergence anomalous segment based on the comparison result between the local outlier factor and the preset outlier threshold. S444. Set the preset position threshold to 2 to 5 and the preset correlation threshold to 0.5 to 0.7, and perform threshold judgment on the divergent abnormal segments. The divergent abnormal segments with the maximum position difference greater than the preset position threshold and the lowest rank correlation coefficient less than the preset correlation threshold are screened as segments to be parsed that have analytical divergence.

[0016] Optionally, S5 specifically includes: S51. For segments to be parsed that have analytical discrepancies, the standard expressions corresponding to the same segment to be parsed in multiple candidate sequences are deduplicated and aggregated, and the deduplicated standard expressions are used to form a set of candidate standard expressions, which serve as the identification framework of the Dempster-Shafer evidence fusion method. S52. Based on the sequence difference analysis results, the position difference of the candidate standard expression in different candidate sequences is reversed and combined with the Kendall rank correlation coefficient between candidate sequences to generate the initial evidence quantity for each agent corresponding to the candidate standard expression. S53. Based on the local outlier identification results, conflict reduction is performed on the initial evidence quantity. When the local outlier corresponding to the segment to be parsed exceeds the preset outlier threshold, the initial evidence quantity of the candidate standard expression in the outlier candidate sequence is reduced, and the reduced evidence quantity is converted into the basic probability allocation value. S54. Using the Dempster-Shafer combination rule, the basic probability assignment values ​​corresponding to different agents are combined. The probability assignment values ​​that point to the same candidate standard expression in the intersection result are accumulated as supporting evidence, and the probability assignment values ​​that have no intersection result are accumulated as the conflict coefficient. S55. Based on the conflict coefficient, the supporting evidence is redistributed to obtain the fusion support of each candidate standard expression. When the highest fusion support and the second highest fusion support meet the preset difference condition, the candidate standard expression corresponding to the highest fusion support is selected as the standardized mapping result.

[0017] Optionally, S6 specifically includes: S61. Based on the standardized mapping results, establish the correspondence between the fragment to be parsed and the standard expression, and replace the standard expression and fill in the related content of the fragment to be parsed according to the fragment order and contextual relationship in the fragment sequence to be parsed, and generate standardized data records. S62. Compare the feedback results with the standardized mapping results, mark the deviations of the changed standard expressions in the feedback results, and generate the mapping deviation amount based on the deviation marks. S63. Using the online mirror descent method, the mapping deviation is mapped to the update step size of the analytical strategy, and the analytical strategy is projected and corrected to obtain the analytical strategy after incremental correction. S64. Based on the incrementally corrected parsing strategy, update the parsing focus parameters corresponding to multiple agents, and use the updated parsing focus parameters to generate candidate sequences for subsequent segments to be parsed. The beneficial effects of this invention are: First, this invention constructs a multi-agent collaborative parsing mechanism for standardized processing, enabling different agents to form differentiated parsing focuses based on the degree of contextual association. Compared with single rule matching or single path semantic recognition, this method can better adapt to the processing scenarios of power industry where multi-source data has inconsistent expressions and strong contextual dependencies.

[0018] Secondly, this invention employs a dual-tower semantic matching method and a gradient boosting tree sorting model to match and prioritize the fragments to be parsed with the power industry standard resource library. It also combines Kendall's rank correlation coefficient and local outlier factors to identify parsing discrepancies in candidate sequences, thereby improving the accuracy and stability of candidate standard expression screening.

[0019] Finally, for segments with analytical discrepancies, this invention employs the Dempster-Shafer evidence fusion method to assess support, and incrementally corrects the analytical strategy using an online mirror descent method based on the feedback results, forming a closed-loop processing mechanism of candidate matching, discrepancy identification, evidence fusion, and feedback correction. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of an intelligent agent parsing and standardization processing method for power industry data proposed in this invention; Figure 2 The flowchart of dual-tower semantic matching and candidate sequence generation is a method for intelligent agent parsing and standardization of power industry data proposed in this invention. Figure 3 This is a flowchart of the candidate sequence difference analysis and parsing divergence segment screening process for an intelligent agent parsing and standardization method for power industry data proposed in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figures 1-3 A method for intelligent agent parsing and standardization of data in the power industry includes the following steps: S1. Acquire multi-source data from the power industry to be processed and preprocess it to form a sequence of segments to be parsed; S2. Construct a multi-agent collaborative parsing mechanism for standardized processing, and configure parsing focus parameters for multiple agents based on the degree of contextual association between each segment in the sequence of segments to be parsed; S3. Each agent uses a dual-tower semantic matching method to perform semantic matching between the sequence of segments to be parsed and the standard resource library of the power industry, based on the parsing focus parameters. The matching priority is adjusted by using a gradient boosting tree sorting model to form multiple candidate sequences. S4. Perform sequence difference analysis on the standard expression in multiple candidate sequences, calculate the position difference of the same standard expression in different candidate sequences, and use Kendall rank correlation coefficient to characterize the order correlation between different candidate sequences. Combine with local outlier factors to screen out segments with analytical discrepancies. S5. For segments to be analyzed that have discrepancies, construct evidence allocation values ​​for candidate standard expressions based on the sequence difference analysis results, and use evidence fusion methods to assess support and screen standardized mapping results. S6. Based on the standardized mapping results, the sequence of segments to be parsed is reconstructed using standard representations to generate standardized data records. Based on the feedback results, the parsing strategy is incrementally corrected using an online mirror descent method.

[0023] In this embodiment, the preprocessing specifically includes: The multi-source data to be processed in the power industry is subjected to structural boundary analysis to identify the boundary positions between data contents with different structural forms, and the multi-source data is split into initial content fragments according to the boundary positions, wherein: Multi-source data includes: data content to be standardized that comes from different power business systems, different data interfaces, or different file structures; The initial content segments are semantically connected. Based on the semantic similarity, contextual co-occurrence relationship and semantic breakpoint position between adjacent initial content segments, the initial content segments are merged or re-segmented to obtain semantic segments. Redundant expressions are compressed and invalid perturbations are removed from semantic segments. Repetitive expressions and anomalous markers that do not participate in the normalized parsing are removed, while the original order relationship between semantic segments is preserved. Based on the contextual position of the semantic fragment and the degree of association between adjacent semantic fragments, a fragment index and context association identifier are configured for the semantic fragment, forming a sequence of fragments to be parsed.

[0024] In this embodiment, the multi-agent collaborative parsing mechanism is a mechanism that differentiates multiple agents based on parsing focus parameters and achieves collaborative parsing through shared fragment states, weight adjustment, and candidate difference feedback. It includes a context-weighted agent, a standard resource-weighted agent, a history-feedback-weighted agent, and a divergence re-judgment agent, wherein: The parsing focus parameter is used to adjust the parsing control of multiple agents during the semantic parsing process, so that different agents form different candidate standard expression generation paths when processing the same segment to be parsed; The context weight agent is used to adjust the scope of context reference for the current segment in the semantic matching process according to the degree of contextual association between adjacent segments in the segment sequence to be parsed; The standard resource weighting agent is used to adjust the candidate retention threshold of the standard expression in the candidate sequence according to the matching distance between the fragment to be parsed and the power industry standard resource library; The historical feedback weighting agent is used to adjust the matching weight of candidate standard expressions corresponding to the same type of fragments according to the hit frequency of the confirmed standardized mapping records. The divergence rematch agent is used to adjust the rematch range of segments with analytic divergence based on the position difference between multiple candidate sequences and local outlier results.

[0025] In this embodiment, S3 specifically includes: S31. Based on the parsing focus parameters corresponding to each agent, the context range of the segments to be parsed in the sequence of segments to be parsed is extracted and weighted, and the segments to be parsed after weighting are semantically encoded to obtain the segment semantic vectors corresponding to each agent. S32. Using a dual-tower semantic matching method, the standard expression items in the power industry standard resource library are semantically encoded to obtain standard semantic vectors. The matching distance between the fragment semantic vector and the standard semantic vector is then measured. The power industry standard resource library is a standardized reference resource used to constrain the conversion of fragments to be parsed into standard expressions, including standard expression items, standard semantic descriptions corresponding to standard expression items, hierarchical relationships between standard expression items, and historically confirmed fragment-standard expression mapping records. Standard semantic descriptions are used to generate standard semantic vectors; Hierarchical affiliation and historically confirmed fragments—standard expression mapping records are used to participate in matching priority adjustment; S33. Based on the matching distance, select the standard expression terms corresponding to each segment to be parsed from near to far to form an initial candidate set. In the initial candidate set, extract the matching distance, contextual relevance, and parsing focus parameters corresponding to each standard expression term as ranking features. S34. Using a gradient boosting tree sorting model, the matching priority of the standard expression items in the initial candidate set is adjusted according to the sorting features, and the standard expression items are arranged according to the adjusted matching priority to form candidate sequences corresponding to multiple agents.

[0026] In this embodiment, the dual-tower semantic matching method specifically includes: Using the segments to be parsed in the sequence as segment-side matching objects and the standard expressions in the power industry standard resource library as standard-side matching objects, semantic vector transformation is performed using both the segment coding tower and the standard coding tower to obtain segment semantic vectors and standard semantic vectors, where: The fragment coding tower and the standard coding tower adopt a semantic coding structure with independent parameters and consistent output dimensions. They perform semantic coding on the fragment-side matching objects and the standard-side matching objects respectively, forming fragment semantic vectors and standard semantic vectors with dimensions ranging from 128 to 768. The standard coding tower generates standard semantic vectors based on standard expression items in the power industry standard resource library, and maps fragment semantic vectors and standard semantic vectors to the same semantic metric space; Based on the degree of contextual relevance of the segment to be parsed and the parameters of parsing focus, the vector weights in the semantic vector of the segment that are related to contextual relevance and standardization tendency are adjusted to form segment matching vectors corresponding to different agents; The cosine distance between the segment matching vector and the standard semantic vector is measured, and the standard expression terms are arranged in ascending order of cosine distance. The standard expression terms that are ranked first are selected as candidate standard expression terms. The number of candidates, K, is determined based on the degree of contextual relevance of the segment to be parsed, where: When the degree of contextual relevance is higher than the preset relevance threshold, K is set to 5 to 10; When the degree of contextual relevance is not higher than the preset relevance threshold, K is between 10 and 20. The candidate criteria expressions are truncated according to the number of candidates K to form an initial candidate set.

[0027] In this embodiment, S34 specifically includes: S341. Form candidate matching pairs by combining the fragment to be parsed with the standard expression terms in the initial candidate set, and construct the ranking feature vector of the candidate matching pairs based on the cosine distance, contextual relevance, and parsing focus parameters obtained from the dual-tower semantic matching. S342. A gradient boosting tree ranking model is adopted to perform round-by-round ranking fitting on the candidate matching pairs. In each round of ranking fitting, a regression tree is constructed based on the ranking residual of the candidate matching pairs in the previous round, and the ranking increment of the candidate matching pairs is output by the regression tree. S343. Accumulate the ranking increments obtained from multiple rounds of ranking fitting to form the ranking score of the candidate matching pair, and adjust the matching priority of the corresponding standard expression item in the initial candidate set according to the ranking score. S344. According to the adjusted matching priority, the standard expression items in the initial candidate set are serialized to form candidate sequences corresponding to multiple agents.

[0028] In this embodiment, S4 specifically includes: S41. For multiple candidate sequences corresponding to the same segment to be parsed, extract the standard expression and its candidate position in each candidate sequence, and use the standard expression that appears in multiple candidate sequences as the rank alignment object to construct the candidate rank table corresponding to the segment to be parsed. S42. Based on the candidate rank table, perform difference operations on the candidate positions of the same standard expression in different candidate sequences to obtain the position difference of the standard expression, and arrange the position difference of each standard expression in a matrix according to the source of the candidate sequence to form a candidate rank difference matrix. S43. Based on the candidate rank difference matrix, any two candidate sequences are paired to form a sequence comparison pair. The Kendall rank correlation coefficient is used to measure the rank correlation of the sequence comparison pair, forming the sequence correlation matrix corresponding to the segment to be parsed. Specifically, this includes: Extract the position records corresponding to any two candidate sequences from the candidate rank difference matrix, and use the standard expressions that appear in both candidate sequences as the rank comparison objects; The standard expression that appears only in one of the candidate sequences is configured as the last position compensation position to form an alignment position table between the two candidate sequences; Based on the alignment position table, the candidate positions of each rank comparison object in the two candidate sequences are extracted, the first position column and the second position column are constructed, and the rank comparison objects are synchronously rearranged according to the ascending order of the first position column to form a position alignment sequence for rank correlation measurement. In a rank-aligned sequence, perform pairwise sequential comparisons on any two rank comparison objects: When the two rank comparison objects have the same order in the first and second columns, they are called a pair of rank pairs in the same direction. When the order is reversed, it is called a reverse rank pair; The Kendall rank correlation coefficient is obtained based on the number of rank pairs in the same direction and the number of rank pairs in opposite directions. The Kendall rank correlation coefficient is used as the ranking correlation value between the two candidate sequences, and the position of the target matrix in the sequence correlation matrix is ​​determined according to the agent number corresponding to the two candidate sequences. The sorting-related values ​​are mapped and filled into the target matrix positions to obtain the sequence correlation matrix of the corresponding segment to be parsed; S44. Based on the candidate rank difference matrix and sequence correlation matrix, construct the divergence vector of the segment to be parsed, and use the local outlier factor method to determine the neighborhood density of the divergence vectors of multiple segments to be parsed. The segments to be parsed whose neighborhood density is lower than that of the surrounding segments to be parsed are identified as segments to be parsed with divergence.

[0029] In this embodiment, S44 specifically includes: S441. Based on the candidate rank difference matrix, the distribution of the position difference of each standard expression in the same segment to be parsed is characterized, the maximum position difference, the median of the position difference, and the proportion of non-zero position differences are determined. Based on the sequence correlation matrix, the lowest rank correlation coefficient is determined, which constitutes the segment divergence coordinates of the segment to be parsed. S442. Using the segment divergence coordinates as the discrimination object, the k-nearest neighbor reachability distance method is used to determine the local neighborhood of each segment to be parsed, and the local reachability density is formed based on the reachability distance between the segment to be parsed and its local neighborhood. S443. Based on the difference in local reachability density between the current segment to be parsed and neighboring segments, generate the local outlier factor of the corresponding segment divergence coordinates, and mark the corresponding segment to be parsed as a divergence anomalous segment based on the comparison result between the local outlier factor and the preset outlier threshold. S444. Set the preset position threshold to 2 to 5 and the preset correlation threshold to 0.5 to 0.7, and perform threshold judgment on the divergent abnormal segments. The divergent abnormal segments with the maximum position difference greater than the preset position threshold and the lowest rank correlation coefficient less than the preset correlation threshold are screened as segments to be parsed that have analytical divergence.

[0030] In this embodiment, S5 specifically includes: S51. For segments to be parsed that have analytical discrepancies, the standard expressions corresponding to the same segment to be parsed in multiple candidate sequences are deduplicated and aggregated, and the deduplicated standard expressions are used to form a set of candidate standard expressions, which serve as the identification framework of the Dempster-Shafer evidence fusion method. S52. Based on the sequence difference analysis results, the position difference of the candidate standard expression in different candidate sequences is reversed and combined with the Kendall rank correlation coefficient between candidate sequences to generate the initial evidence quantity for each agent corresponding to the candidate standard expression. S53. Based on the local outlier identification results, conflict reduction is performed on the initial evidence quantity. When the local outlier corresponding to the segment to be parsed exceeds the preset outlier threshold, the initial evidence quantity of the candidate standard expression in the outlier candidate sequence is reduced, and the reduced evidence quantity is converted into the basic probability allocation value. S54. Using the Dempster-Shafer combination rule, the basic probability assignment values ​​corresponding to different agents are combined. The probability assignment values ​​that point to the same candidate standard expression in the intersection result are accumulated as supporting evidence, and the probability assignment values ​​with no intersection result are accumulated as the conflict coefficient, where: The Dempster-Shafer combination rule is as follows: within the candidate standard expression set, the basic probability assignment values ​​generated by any two agents are paired and combined, and it is determined one by one whether the two candidate pointers in the pair fall into the same candidate standard expression. When two candidate pointers fall into the same candidate standard expression, the corresponding probability allocation product is included in the supporting evidence of that candidate standard expression; When two candidate pointers fall into different candidate standard expressions and there is no common candidate pointer, the corresponding probability allocation product is included in the conflict evidence; After completing all pairwise combinations, the supporting evidence for each candidate criterion is redistributed based on the proportion of conflicting evidence to all combined evidence, thus obtaining the fusion support. S55. Based on the conflict coefficient, the supporting evidence is redistributed to obtain the fusion support of each candidate standard expression. When the highest fusion support and the second highest fusion support meet the preset difference condition, the candidate standard expression corresponding to the highest fusion support is selected as the standardized mapping result.

[0031] In this embodiment, S6 specifically includes: S61. Based on the standardized mapping results, establish the correspondence between the fragment to be parsed and the standard expression, and replace the standard expression and fill in the related content of the fragment to be parsed according to the fragment order and contextual relationship in the fragment sequence to be parsed, and generate standardized data records. S62. Compare the feedback results with the standardized mapping results, mark the deviations of the changed standard expressions in the feedback results, and generate the mapping deviation amount based on the deviation marks. S63. Using the online mirror descent method, the mapping deviation is mapped to the update step size of the analytical strategy, and the analytical strategy is projected and corrected to obtain the analytical strategy after incremental correction. S64. Based on the parsing strategy after incremental correction, update the parsing focus parameters corresponding to multiple agents, and use the updated parsing focus parameters to generate candidate sequences for subsequent segments to be parsed.

[0032] Example 1: To verify the feasibility of this invention in practice, it was applied to a data standardization processing scenario in the power industry. In this scenario, the data sources are scattered, the structural forms are inconsistent, and the same standard meaning has multiple expressions in different data contents. Some fragments contain abbreviations, synonyms, non-standard descriptions, and context-dependent descriptions. When using traditional field mapping or keyword rules, it is easy to incorrectly merge content with similar semantics but different standard meanings, and after new expressions appear, the rules need to be manually maintained repeatedly, resulting in insufficient stability of the standardization results.

[0033] In this scenario, multi-source data is first preprocessed to remove invalid perturbations and redundant expressions, and a sequence of segments to be parsed is formed based on semantic boundaries and the degree of contextual relevance. For segments with high contextual relevance, a smaller contextual reference range is retained; for segments with low contextual relevance, the contextual reference range is expanded. Subsequently, a multi-agent collaborative parsing mechanism is constructed, and parsing focus parameters are configured for different agents based on the degree of contextual relevance between segments, enabling multiple agents to perform candidate standard expression matching for the same segment from different parsing focuses.

[0034] During candidate generation, each agent employs a dual-tower semantic matching method to map the segment to be parsed and the standard expressions in the power industry standard resource library to the same semantic metric space, and forms an initial candidate set based on cosine distance. For segments with high contextual relevance, the number of candidates K is set to 5 to 10; for segments with low contextual relevance, K is set to 10 to 20. Subsequently, a gradient boosting tree ranking model is used to adjust the matching priority of the candidate standard expressions, forming a candidate sequence.

[0035] For segments with significant differences among candidate sequences, Kendall's rank correlation coefficient and local outlier factor are used to identify and resolve discrepancies. The Dempster-Shafer evidence fusion method is then applied to assess the support of these discrepancies, and standardized mapping results are selected. Finally, based on the standardized mapping results, the segments to be resolved are reconstructed using standard representations, generating standardized data records. Based on feedback results, an online mirror descent method is used to incrementally correct the resolution strategy.

[0036] During the validation process, 12,000 data entries were selected for processing. After preprocessing, 38,640 fragments were generated to be parsed. The standard resource library contained 4,280 standard expressions. The standardization accuracy rate of the traditional keyword matching method was 86.7%, with a manual review rate of 16.5%. The standardization accuracy rate of the ordinary single-model semantic matching method was 91.4%, with a manual review rate of 9.8%.

[0037] After adopting the method of this invention, the standardization accuracy reached 96.2%, the first candidate hit rate reached 94.6%, the proportion of parsed divergent fragments decreased to 6.8%, and the proportion of manual review decreased to 4.7%. After three rounds of feedback correction, the duplicate mismatch rate of similar fragments decreased from 7.6% to 2.1%, indicating that this invention can maintain high stability in standardization processing under conditions of inconsistent multi-source expressions, similar candidate standard expressions, and strong contextual dependence.

[0038] Table 1 Comparison of Data Standardization Processing Effects in the Power Industry

[0039] As can be seen from the data in Table 1, the method of the present invention outperforms traditional keyword matching and single-model semantic matching in terms of standardization accuracy, candidate hit effect, parsing divergence control, and manual review requirements. This indicates that the present invention, through multi-agent collaborative parsing, candidate sequence difference analysis, and evidence fusion processing, can more stably complete the standardization processing of multi-source data in the power industry, while reducing repeated mismatches and maintaining good processing efficiency.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An agent parsing and standardization processing method for power industry data, characterized in that, Includes the following steps: S1. Acquire multi-source data from the power industry to be processed and preprocess it to form a sequence of segments to be parsed; S2. Construct a multi-agent collaborative parsing mechanism for standardized processing, and configure parsing focus parameters for multiple agents based on the degree of contextual association between each segment in the sequence of segments to be parsed; S3. Each agent uses a dual-tower semantic matching method to perform semantic matching between the sequence of segments to be parsed and the standard resource library of the power industry, based on the parsing focus parameters. The matching priority is adjusted by using a gradient boosting tree sorting model to form multiple candidate sequences. S4. Perform sequence difference analysis on the standard expression in multiple candidate sequences, calculate the position difference of the same standard expression in different candidate sequences, and use Kendall rank correlation coefficient to characterize the order correlation between different candidate sequences. Combine with local outlier factors to screen out segments with analytical discrepancies. S5. For segments to be analyzed that have discrepancies, construct evidence allocation values ​​for candidate standard expressions based on the sequence difference analysis results, and use evidence fusion methods to assess support and screen standardized mapping results. S6. Based on the standardized mapping results, the sequence of segments to be parsed is reconstructed using standard representations to generate standardized data records. Based on the feedback results, the parsing strategy is incrementally corrected using an online mirror descent method.

2. The method of claim 1, wherein the method comprises: The preprocessing specifically includes: The multi-source data to be processed in the power industry is subjected to structural boundary analysis to identify the boundary positions between data contents with different structural forms, and the multi-source data is split into initial content fragments according to the boundary positions, wherein: The multi-source data includes: data content to be standardized from different power business systems, different data interfaces, or different file structures; The initial content segments are semantically connected. Based on the semantic similarity, contextual co-occurrence relationship and semantic breakpoint position between adjacent initial content segments, the initial content segments are merged or re-segmented to obtain semantic segments. Redundant expressions are compressed and invalid perturbations are removed from semantic segments. Repetitive expressions and anomalous markers that do not participate in the normalized parsing are removed, while the original order relationship between semantic segments is preserved. Based on the contextual position of the semantic fragment and the degree of association between adjacent semantic fragments, a fragment index and context association identifier are configured for the semantic fragment, forming a sequence of fragments to be parsed.

3. The method of claim 1, wherein the method comprises: The multi-agent collaborative parsing mechanism is a mechanism that applies differentiated constraints to multiple agents based on parsing focus parameters, and achieves collaborative parsing through shared fragment states, weight adjustment, and candidate difference feedback. It includes a context-weighted agent, a standard resource-weighted agent, a historical feedback-weighted agent, and a divergence re-judgment agent, wherein: The parsing focus parameters are used to adjust the parsing control quantities of multiple agents during the semantic parsing process, so that different agents form different candidate standard expression generation paths when processing the same segment to be parsed. The context weight agent is used to adjust the scope of context reference for the current segment in the semantic matching process according to the degree of contextual association between adjacent segments in the segment sequence to be parsed; The standard resource weighting agent is used to adjust the candidate retention threshold of the standard expression in the candidate sequence according to the matching distance between the fragment to be parsed and the power industry standard resource library; The historical feedback weighting agent is used to adjust the matching weight of candidate standard expressions corresponding to the same type of fragments according to the hit frequency of the confirmed standardized mapping records. The divergence rematch agent is used to adjust the rematch range of segments with analytic divergence based on the position difference between multiple candidate sequences and local outlier results.

4. The method of claim 1, wherein the method comprises: S3 specifically includes: S31. Based on the parsing focus parameters corresponding to each agent, the context range of the segments to be parsed in the sequence of segments to be parsed is extracted and weighted, and the segments to be parsed after weighting are semantically encoded to obtain the segment semantic vectors corresponding to each agent. S32. Using the dual-tower semantic matching method, the standard expression items in the power industry standard resource library are semantically encoded to obtain standard semantic vectors, and the matching distance between the fragment semantic vector and the standard semantic vector is measured. S33. Based on the matching distance, select the standard expression terms corresponding to each segment to be parsed from near to far to form an initial candidate set. In the initial candidate set, extract the matching distance, contextual relevance, and parsing focus parameters corresponding to each standard expression term as ranking features. S34. Using a gradient boosting tree sorting model, the matching priority of the standard expression items in the initial candidate set is adjusted according to the sorting features, and the standard expression items are arranged according to the adjusted matching priority to form candidate sequences corresponding to multiple agents.

5. The power industry data-oriented agent parsing and standardization processing method according to claim 4, characterized in that, The dual-tower semantic matching method specifically includes: Using the segments to be parsed in the sequence as segment-side matching objects and the standard expressions in the power industry standard resource library as standard-side matching objects, semantic vector transformation is performed using both the segment coding tower and the standard coding tower to obtain segment semantic vectors and standard semantic vectors, where: The fragment coding tower and the standard coding tower adopt a semantic coding structure with independent parameters and consistent output dimensions. They perform semantic coding on the fragment-side matching objects and the standard-side matching objects respectively, forming fragment semantic vectors and standard semantic vectors with dimensions ranging from 128 to 768. The standard coding tower generates standard semantic vectors based on standard expression items in the power industry standard resource library, and maps fragment semantic vectors and standard semantic vectors to the same semantic metric space; Based on the degree of contextual relevance of the segment to be parsed and the parameters of parsing focus, the vector weights in the semantic vector of the segment that are related to contextual relevance and standardization tendency are adjusted to form segment matching vectors corresponding to different agents; The cosine distance between the segment matching vector and the standard semantic vector is measured, and the standard expression terms are arranged in ascending order of cosine distance. The standard expression terms that are ranked first are selected as candidate standard expression terms. The number of candidates, K, is determined based on the degree of contextual relevance of the segment to be parsed, where: When the degree of contextual relevance is higher than the preset relevance threshold, K is set to 5 to 10; When the degree of contextual relevance is not higher than the preset relevance threshold, K is between 10 and 20. The candidate criteria expressions are truncated according to the number of candidates K to form an initial candidate set.

6. The method of claim 4, wherein the method comprises: S34 specifically includes: S341. Form candidate matching pairs by combining the fragment to be parsed with the standard expression terms in the initial candidate set, and construct the ranking feature vector of the candidate matching pairs based on the cosine distance, contextual relevance, and parsing focus parameters obtained from the dual-tower semantic matching. S342. A gradient boosting tree ranking model is adopted to perform round-by-round ranking fitting on the candidate matching pairs. In each round of ranking fitting, a regression tree is constructed based on the ranking residual of the candidate matching pairs in the previous round, and the ranking increment of the candidate matching pairs is output by the regression tree. S343. Accumulate the ranking increments obtained from multiple rounds of ranking fitting to form the ranking score of the candidate matching pair, and adjust the matching priority of the corresponding standard expression item in the initial candidate set according to the ranking score. S344. According to the adjusted matching priority, the standard expression items in the initial candidate set are serialized to form candidate sequences corresponding to multiple agents.

7. The method for intelligent agent parsing and standardization of power industry data according to claim 1, characterized in that, S4 specifically includes: S41. For multiple candidate sequences corresponding to the same segment to be parsed, extract the standard expression and its candidate position in each candidate sequence, and use the standard expression that appears in multiple candidate sequences as the rank alignment object to construct the candidate rank table corresponding to the segment to be parsed. S42. Based on the candidate rank table, perform difference operations on the candidate positions of the same standard expression in different candidate sequences to obtain the position difference of the standard expression, and arrange the position difference of each standard expression in a matrix according to the source of the candidate sequence to form a candidate rank difference matrix. S43. Based on the candidate rank difference matrix, any two candidate sequences are used to form a sequence comparison pair, and the Kendall rank correlation coefficient is used to measure the rank correlation of the sequence comparison pair to form the sequence correlation matrix of the corresponding segment to be parsed. S44. Based on the candidate rank difference matrix and sequence correlation matrix, construct the divergence vector of the segment to be parsed, and use the local outlier factor method to determine the neighborhood density of the divergence vectors of multiple segments to be parsed. The segments to be parsed whose neighborhood density is lower than that of the surrounding segments to be parsed are identified as segments to be parsed with divergence.

8. The method for intelligent agent parsing and standardization processing of power industry data according to claim 7, characterized in that, S44 specifically includes: S441. Based on the candidate rank difference matrix, the distribution of the position difference of each standard expression in the same segment to be parsed is characterized, the maximum position difference, the median of the position difference, and the proportion of non-zero position differences are determined. Based on the sequence correlation matrix, the lowest rank correlation coefficient is determined, which constitutes the segment divergence coordinates of the segment to be parsed. S442. Using the segment divergence coordinates as the discrimination object, the k-nearest neighbor reachability distance method is used to determine the local neighborhood of each segment to be parsed, and the local reachability density is formed based on the reachability distance between the segment to be parsed and its local neighborhood. S443. Based on the difference in local reachability density between the current segment to be parsed and neighboring segments, generate the local outlier factor of the corresponding segment divergence coordinates, and mark the corresponding segment to be parsed as a divergence anomalous segment based on the comparison result between the local outlier factor and the preset outlier threshold. S444. Set the preset position threshold to 2 to 5 and the preset correlation threshold to 0.5 to 0.7, and perform threshold judgment on the divergent abnormal segments. The divergent abnormal segments with the maximum position difference greater than the preset position threshold and the lowest rank correlation coefficient less than the preset correlation threshold are screened as segments to be parsed that have analytical divergence.

9. The method for intelligent agent parsing and standardization of power industry data according to claim 1, characterized in that, S5 specifically includes: S51. For segments to be parsed that have analytical discrepancies, the standard expressions corresponding to the same segment to be parsed in multiple candidate sequences are deduplicated and aggregated, and the deduplicated standard expressions are used to form a set of candidate standard expressions, which serve as the identification framework of the Dempster-Shafer evidence fusion method. S52. Based on the sequence difference analysis results, the position difference of the candidate standard expression in different candidate sequences is reversed and combined with the Kendall rank correlation coefficient between candidate sequences to generate the initial evidence quantity for each agent corresponding to the candidate standard expression. S53. Based on the local outlier identification results, conflict reduction is performed on the initial evidence quantity. When the local outlier corresponding to the segment to be parsed exceeds the preset outlier threshold, the initial evidence quantity of the candidate standard expression in the outlier candidate sequence is reduced, and the reduced evidence quantity is converted into the basic probability allocation value. S54. Using the Dempster-Shafer combination rule, the basic probability assignment values ​​corresponding to different agents are combined. The probability assignment values ​​that point to the same candidate standard expression in the intersection result are accumulated as supporting evidence, and the probability assignment values ​​that have no intersection result are accumulated as the conflict coefficient. S55. Based on the conflict coefficient, the supporting evidence is redistributed to obtain the fusion support of each candidate standard expression. When the highest fusion support and the second highest fusion support meet the preset difference condition, the candidate standard expression corresponding to the highest fusion support is selected as the standardized mapping result.

10. The method for intelligent agent parsing and standardization of power industry data according to claim 1, characterized in that, S6 specifically includes: S61. Based on the standardized mapping results, establish the correspondence between the fragment to be parsed and the standard expression, and replace the standard expression and fill in the related content of the fragment to be parsed according to the fragment order and contextual relationship in the fragment sequence to be parsed, and generate standardized data records. S62. Compare the feedback results with the standardized mapping results, mark the deviations of the changed standard expressions in the feedback results, and generate the mapping deviation amount based on the deviation marks. S63. Using the online mirror descent method, the mapping deviation is mapped to the update step size of the analytical strategy, and the analytical strategy is projected and corrected to obtain the analytical strategy after incremental correction. S64. Based on the parsing strategy after incremental correction, update the parsing focus parameters corresponding to multiple agents, and use the updated parsing focus parameters to generate candidate sequences for subsequent segments to be parsed.

Images