A machine learning based p&id symbol potential location inference method

Abstract

The application discloses a P&ID symbol potential position inference method based on machine learning, comprising the following steps: obtaining a drawing image and text data, completing pretreatment and vectorization construction process diagram structure; using a symbol position inference model to generate a candidate set and output a prediction distribution and confidence; dividing the candidate set into a global topology layer and a local candidate layer, calculating mutual information and fusing into a hierarchical index; running a BatchBALD algorithm based on the hierarchical index to generate a ranked candidate set; adopting a two-stage screening strategy to form an accurate set; performing constraint projection in the accurate set and combining k-DPP sampling to generate a batch query set; introducing a memory unit in the batch query process and updating the model combined with artificial labeling to output symbol position coordinates and category results. The application combines information gain ranking and constraint sampling, and improves the accuracy and robustness of symbol position inference.

Description

Technical Field

[0001] This invention relates to the field of industrial process design and intelligent identification technology, and in particular to a method for inferring the latent location of P&ID symbols based on machine learning. Background Technology

[0002] In industrial process design and engineering implementation, P&ID drawings are widely used in engineering design, operation management and maintenance as an important carrier for expressing process logic and equipment connection relationships. Their content usually consists of symbols, lines and text descriptions, used to represent instruments, equipment, pipeline flow direction and parameter information. Therefore, how to efficiently and accurately infer the potential location of symbols in drawings has become a key task for intelligent design and decision support. Existing methods mostly rely on manual annotation or rule-based detection methods. This approach relies too much on human experience and template matching, and it is difficult to meet the dual requirements of efficiency and accuracy when processing large-scale drawings.

[0003] In image processing and symbol detection, traditional research mostly uses edge detection, template matching, and morphological operations. These methods can identify common symbols in drawings with a high degree of standardization, but the detection effect drops significantly when there is noise in the drawings, or when the symbols have scale changes or shape distortions. With the development of deep learning methods, convolutional neural networks have been introduced into symbol recognition, which can extract local features and perform classification tasks. Although the recognition ability has been improved, it is still at the symbol detection stage and lacks accurate inference of symbol positions. In particular, it is insufficient in the modeling of topological structure and process logic, which can easily lead to recognition results that do not match the actual logic.

[0004] In the process of generating and screening candidate symbol positions, existing technologies typically rely on confidence thresholds to screen candidate positions. This approach fails to fully express the uncertainty of candidate positions and makes it difficult to quantify the correlation between candidate positions. Some methods attempt to use probability distributions to characterize candidate positions, but they still rely mainly on independent ranking and ignore global joint information. In the screening stage, most methods adopt a single rule decision, and the results are easily affected by error propagation, which reduces the overall accuracy. As for information selection methods, existing studies mostly rely on single-sample uncertainty sampling, such as maximum entropy and minimum confidence methods. These strategies can only reflect the uncertainty of individual samples and ignore the complementarity and joint information gain between batch samples. Therefore, they are difficult to play their due role in large-scale candidate sets.

[0005] In terms of symbol position adjustment and constraint modeling, existing methods often lack systematic utilization of the constraints specific to P&ID drawings. Symbol spacing, alignment, and pipeline connectivity are key to ensuring the rationality of position inference, but traditional methods lack a mechanism to incorporate these factors into the overall calculation. Even if there are reasonable positions in the candidate set, they may be misjudged because they do not meet the process logic requirements. In the active learning and model update stages, most methods adopt single-round iterative optimization without introducing a memory mechanism to quantify the contribution of historical samples, resulting in large fluctuations in the model training process and making it difficult to guarantee the stability and consistency of the final inference results.

[0006] Therefore, how to provide a machine learning-based method for inferring the latent location of P&ID symbols is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose a machine learning-based method for inferring the latent location of P&ID symbols. This invention constructs a process diagram structure, introduces a symbol location inference model, combines hierarchical mutual information index and BatchBALD algorithm, adopts a two-stage screening strategy, performs constraint projection and k-DPP sampling, and combines memory units and manual annotation feedback to achieve continuous optimization of the symbol location inference model. The invention details the algorithm flow for efficiently generating latent symbol location coordinates and symbol categories in complex process drawings, and has the advantages of high recognition accuracy, strong adaptability to process logic constraints, and stable model iterative optimization capability.

[0008] A machine learning-based method for inferring the latent location of P&ID symbols according to an embodiment of the present invention includes the following steps:

[0009] Acquire P&ID drawing images and text data, perform preprocessing and vectorization, and construct the process diagram structure;

[0010] In the process diagram structure, a set of candidate symbol positions is generated by the symbol position inference model, and a prediction distribution and confidence level are output for each candidate symbol position;

[0011] The candidate symbol location set is divided into a global topological layer and a local candidate layer. Mutual information is calculated separately and then fused into a hierarchical mutual information index.

[0012] The BatchBALD algorithm is run based on the hierarchical mutual information index to generate a mutual information sorted set with joint information gain sorting.

[0013] A two-stage screening strategy is adopted for the mutual information ranking set. In the first stage, a pre-screening set is formed based on the proxy indicator. In the second stage, the mutual information ranking is updated on the pre-screening set to obtain the precise screening set.

[0014] Perform constraint projection operations on the precise filter set to map the positions of candidate symbols that do not meet the constraints of symbol spacing, alignment and pipeline connectivity to the feasible solution set, and use k-DPP sampling on the feasible solution set to generate a batch query sample set.

[0015] In the process of generating a batch query sample set, a memory unit is introduced to adjust the candidate weights based on the contribution of historically selected samples, and the symbol position inference model is updated in combination with the results of manual annotation, outputting the potential position coordinates of P&ID symbols and the inference results of symbol categories.

[0016] Optionally, the construction of the process diagram structure specifically includes:

[0017] Acquire P&ID drawing images and text data, perform noise reduction, distortion correction and grayscale normalization on the drawing images, and generate preprocessed images;

[0018] The text data is segmented, character recognized, and structured parsed to generate a set of text features;

[0019] Edge detection and line extraction are performed on the preprocessed image to generate a set of connected lines;

[0020] Symbol detection is performed on the preprocessed image to generate a symbol set, where each symbol contains its center coordinates;

[0021] The connection set and symbol set are vectorized and combined with the text feature set to generate a process diagram structure. The process diagram structure contains a node set and an edge set. The node set corresponds to the symbol set and the text feature set, and the edge set corresponds to the connection set.

[0022] Optionally, the generation and prediction distribution of the candidate symbol location set and the confidence output specifically include:

[0023] Load the symbol position inference model into the process diagram structure, and input the node set and edge set into the symbol position inference model;

[0024] In the symbol position inference model, the features of the node set and edge set are encoded by a graph encoder to generate node feature vectors and edge feature vectors;

[0025] In the symbol position inference model, node feature vectors and edge feature vectors are fused by an aggregation module to generate a graph structure embedding representation;

[0026] In the symbol location inference model, a preliminary set of candidate symbol locations is generated based on graph structure embedding representation using a spatial decoder. Each candidate symbol location in the preliminary set contains the horizontal coordinate, the vertical coordinate, and the category label.

[0027] The filtering module is run on the initial set of candidate symbol positions to perform spatial nonmaximum suppression and minimum spacing constraint filtering, thereby generating a set of candidate symbol positions.

[0028] In the symbol location inference model, the classification branch outputs the category probability distribution of the candidate symbol location set, and the regression branch outputs the location coordinate confidence distribution of the candidate symbol location set.

[0029] Add graph index information to each candidate symbol position in the candidate symbol position set. The graph index information includes the node identifier, adjacent edge identifiers, and coordinates in the graph.

[0030] Output the set of candidate symbol locations, the category probability distribution, and the confidence distribution of location coordinates.

[0031] Optionally, the calculation of the hierarchical mutual information index specifically includes:

[0032] Read the candidate symbol location set, category probability distribution, and location coordinate confidence distribution;

[0033] Based on the topological relationships in the process diagram structure and the spatial neighborhood relationships of the candidate symbol positions, the set of candidate symbol positions is divided into a global topological layer candidate set and a local candidate layer candidate set.

[0034] A joint probability distribution and marginal distribution are established between the global topology layer candidate set and the class probability distribution and the location coordinate confidence distribution; a joint probability distribution and marginal distribution are also established between the local candidate layer candidate set and the class probability distribution and the location coordinate confidence distribution.

[0035] Based on the definition of mutual information in information theory, the logarithm of the ratio of the joint probability distribution to the marginal distribution is taken and then summed with weights to form a hierarchical mutual information index. Recorded as:

[0036] ;

[0037] in, This is the global topology layer candidate set. For the local candidate layer candidate set, For a set of symbol categories, For a set of position coordinates, The position of the candidate symbol for the global topology layer. This represents the position of candidate symbols in the local candidate layer. For symbol categories, The position coordinate vector, The joint probability distribution of the global topology layer. The joint probability distribution of the local candidate layers. The edge distribution of candidate symbol positions in the global topology layer. The edge distribution of candidate symbol positions in the local candidate layer. The joint marginal distribution of category and location coordinates, and Let be the weighting coefficient, satisfying , It is the natural logarithm.

[0038] Optionally, the generation of the mutual information sorting set specifically includes:

[0039] Read the candidate symbol location set, category probability distribution, location coordinate confidence distribution, and hierarchical mutual information index;

[0040] Generate a prediction distribution for each candidate symbol location in the candidate symbol location set. The prediction distribution consists of a class probability distribution and a location coordinate confidence distribution.

[0041] Monte Carlo sampling is performed on the prediction distribution to construct a set of parameter samples. The category prediction distribution of candidate symbol positions is calculated under each parameter sample. Given the parameter samples, the batch output conditional independence assumption is adopted, and the batch joint prediction distribution is represented as a product of the prediction distributions of individual candidate symbol positions.

[0042] Based on the definition in information theory that mutual information equals the difference between entropy and conditional entropy, and combining the conditional independence assumption and the Monte Carlo approximation, the joint information gain estimate of BatchBALD is calculated as follows:

[0043] ;

[0044] in, This represents the Monte Carlo estimate of the joint information gain of BatchBALD. " indicates an estimation operator, " " represents the mutual information operator, This represents a random variable composed of the classification outputs of candidate symbol positions within a batch. Indicates batch size. Indicates the model parameters inferred from the sign position. Indicates the first One parameter sample, The parameter is the sample index and its value range is... arrive , Indicates the number of parameter samples. Represents a set of symbol categories. Indicates length is A set of category sequences Represents a category sequence, The index of the candidate symbol position within the batch and the range of values ​​is arrive , Indicates in parameter samples Next The category prediction distribution of each candidate symbol position. This represents the batch joint prediction distribution obtained by averaging over the parameter sample set;

[0045] Based on the joint information gain estimation and the hierarchical mutual information index, the candidate symbol position set is combined and sorted according to the unified sorting criterion to generate a mutual information sorted set.

[0046] Optionally, the execution of the two-stage screening strategy specifically includes:

[0047] Perform a two-stage filtering strategy in the mutually information-ranked set;

[0048] In the first stage, the category probability distribution, location coordinate confidence distribution, and hierarchical mutual information index of the candidate symbol location set are read. A surrogate index value is calculated for each candidate symbol location. The surrogate index value is obtained by weighting the category probability distribution score, the location coordinate confidence distribution score, and the hierarchical mutual information index. Candidate symbol locations with surrogate index values ​​not less than the threshold are formed into a pre-screening set.

[0049] In the second stage, the joint information gain estimation of BatchBALD is called within the pre-screened set to calculate the ranking score for each candidate symbol position in the pre-screened set, and the ranking order is adjusted in combination with the corresponding hierarchical mutual information index to form the precise screening set.

[0050] Optionally, the execution of the constraint projection and k-DPP sampling specifically includes:

[0051] Perform constraint projection operations in the precise selection set to adjust the positions of candidate symbols that do not meet the symbol spacing constraint, alignment constraint, and pipeline connectivity constraint into the feasible solution set;

[0052] The symbol spacing constraint is determined by calculating the sum of the squares of the differences between the x-coordinates and y-coordinates between any two candidate symbol positions. When the result is less than a preset threshold, the candidate symbol position is moved in the coordinate plane until the spacing threshold is met.

[0053] Alignment constraints are determined by detecting the angle between the center coordinates of the candidate symbol position and the pipeline direction. When the angle is not equal to zero degrees or ninety degrees, the candidate symbol position is adjusted along the nearest direction until the center coordinates are horizontally or vertically aligned with the pipeline.

[0054] Pipeline connectivity constraints are determined by measuring the Euclidean distance between the candidate symbol position and the pipeline endpoint or pipeline midpoint. When the distance is greater than the preset radius, the candidate symbol position is moved to the boundary of the circular neighborhood range.

[0055] Construct a kernel matrix in the feasible solution set. Each element in the kernel matrix is ​​obtained by the inner product of the feature vectors of the candidate symbol positions. The feature vectors include geometric position, class encoding and adjacency relation.

[0056] The k-DPP sampling algorithm is run on the kernel matrix to select a fixed number of candidate symbol positions from the feasible solution set and generate a batch query sample set.

[0057] Optionally, the introduction of the memory unit and model update specifically include:

[0058] A memory unit is introduced during the batch query sample set generation process. The memory unit stores sample identifiers, feature vectors, graph index information, selection rounds, annotation status, contribution scores, and sampling weights.

[0059] Read the selected sample records from the history, and for each record, calculate the training loss reduction, cumulative gradient path value, and time interval between selection rounds, calculate the contribution score, and write it into the memory unit;

[0060] The contribution score is mapped to the sampling weight adjustment amount, and the candidate symbol position is matched with the memory unit entry and the sampling weight is updated according to the index in the batch query sample set.

[0061] Based on the updated sampling weights, resampling is performed within the batch query sample set to generate a labeled list;

[0062] Submit the annotation list for manual annotation, and collect the annotation results of symbol categories and location coordinates;

[0063] Based on the graph index information, a one-to-one correspondence is established between the annotation results and the candidate symbol position set, training sample entries are generated and written into the training dataset;

[0064] Based on the training dataset, a parameter update process is performed to update the symbol position inference model parameters and synchronously update the label status and contribution score in the memory unit.

[0065] Output the potential location coordinates of P&ID symbols and the inference results of symbol categories, and save the location confidence records.

[0066] The beneficial effects of this invention are:

[0067] The machine learning-based P&ID symbol latent location inference method proposed in this invention can simultaneously utilize image features, text features, and topological structure information in complex process drawings, solving the problem of inconsistency between symbol location inference and process logic caused by existing methods relying only on local visual features. By introducing a symbol location inference model and outputting the category probability distribution and location coordinate confidence distribution during the candidate generation stage, the representation of symbol location is more comprehensive, effectively improving the quality of the candidate location set.

[0068] In the information selection stage, this invention utilizes a combination of hierarchical mutual information metrics and the BatchBALD algorithm. This approach not only measures the uncertainty of individual candidate symbol positions but also assesses the joint information gain among batch candidate positions, thus fully reflecting the global optimization effect during the selection process. Through a two-stage selection strategy, pre-screening is performed using surrogate metrics, followed by updating the ranking using joint information gain and hierarchical mutual information metrics. This ensures that the precise selection set better aligns with actual process logic requirements. Furthermore, the projection mechanism incorporating symbol spacing, alignment, and pipeline connectivity constraints guarantees the rationality of the selection results in both geometric space and process logic.

[0069] During model optimization, this invention introduces a memory unit to record and utilize the contribution of historically selected samples. By dynamically adjusting sampling weights, the batch query sample set can take into account both new and historical information, and continuous iterative optimization of the model is achieved based on feedback from manual annotation results. Through this closed-loop mechanism, the model has been significantly improved in terms of accuracy, stability, and scalability, and can adapt to the automated processing needs of large-scale P&ID drawings. Attached Figure Description

[0070] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0071] Figure 1 This is a flowchart of a machine learning-based P&ID symbol latent location inference method proposed in this invention;

[0072] Figure 2 This is a schematic diagram of the process diagram structure construction for a machine learning-based P&ID symbol latent location inference method proposed in this invention.

[0073] Figure 3 This is a schematic diagram of the BatchBALD sorting mechanism of a machine learning-based P&ID symbol latent position inference method proposed in this invention. Detailed Implementation

[0074] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0075] refer to Figure 1-3 A machine learning-based method for inferring the latent location of P&ID symbols includes the following steps:

[0076] Acquire P&ID drawing images and text data, perform preprocessing and vectorization, and construct the process diagram structure;

[0077] In the process diagram structure, a set of candidate symbol positions is generated by the symbol position inference model, and a prediction distribution and confidence level are output for each candidate symbol position;

[0078] The candidate symbol location set is divided into a global topological layer and a local candidate layer. Mutual information is calculated separately and then fused into a hierarchical mutual information index.

[0079] The BatchBALD algorithm is run based on the hierarchical mutual information index to generate a mutual information sorted set with joint information gain sorting.

[0080] A two-stage screening strategy is adopted for the mutual information ranking set. In the first stage, a pre-screening set is formed based on the proxy indicator. In the second stage, the mutual information ranking is updated on the pre-screening set to obtain the precise screening set.

[0081] Perform constraint projection operations on the precise filter set to map the positions of candidate symbols that do not meet the constraints of symbol spacing, alignment and pipeline connectivity to the feasible solution set, and use k-DPP sampling on the feasible solution set to generate a batch query sample set.

[0082] In the process of generating a batch query sample set, a memory unit is introduced to adjust the candidate weights based on the contribution of historically selected samples, and the symbol position inference model is updated in combination with the results of manual annotation, outputting the potential position coordinates of P&ID symbols and the inference results of symbol categories.

[0083] This invention achieves end-to-end intelligent inference from original drawings and text data to symbol positions and categories by constructing a process diagram structure in P&ID drawings and combining a symbol position inference model, hierarchical mutual information index, BatchBALD algorithm, two-stage screening strategy, constrained projection and k-DPP sampling, and a closed-loop update mechanism of memory cells and manual annotations. It can effectively reduce the number of candidates, improve the accuracy of symbol recognition, and achieve continuous adaptive optimization of the model while ensuring reasonable symbol spacing, alignment, and pipeline connectivity. This improves the automation level of P&ID drawing parsing and the practicality of industrial applications.

[0084] In this embodiment, the construction of the process diagram structure specifically includes:

[0085] Acquire P&ID drawing images and text data, perform noise reduction, distortion correction and grayscale normalization on the drawing images, and generate preprocessed images;

[0086] The text data is segmented, character recognized, and structured parsed to generate a set of text features;

[0087] Edge detection and line extraction are performed on the preprocessed image to generate a set of connected lines;

[0088] Symbol detection is performed on the preprocessed image to generate a symbol set, where each symbol contains its center coordinates;

[0089] The connection set and symbol set are vectorized and combined with the text feature set to generate a process diagram structure. The process diagram structure contains a node set and an edge set. The node set corresponds to the symbol set and the text feature set, and the edge set corresponds to the connection set.

[0090] In the process drawing structure construction stage, this invention uses multiple steps, including image denoising, distortion correction, grayscale normalization, edge detection, line extraction, and symbol detection, to vectorize and fuse the structural information and text features in the drawing image, forming a set of nodes and edges, which are then uniformly mapped into the process drawing structure. This provides a clear graph structure input for subsequent symbol position inference, reduces the impact of image noise, text interference, and geometric distortion, and ensures the complete expression and semantic consistency of drawing elements.

[0091] In this embodiment, the generation and prediction distribution of the candidate symbol position set and the confidence output specifically include:

[0092] Load the symbol position inference model into the process diagram structure, and input the node set and edge set into the symbol position inference model;

[0093] In the symbol position inference model, the features of the node set and edge set are encoded by a graph encoder to generate node feature vectors and edge feature vectors;

[0094] In the symbol position inference model, node feature vectors and edge feature vectors are fused by an aggregation module to generate a graph structure embedding representation;

[0095] In the symbol location inference model, a preliminary set of candidate symbol locations is generated based on graph structure embedding representation using a spatial decoder. Each candidate symbol location in the preliminary set contains the horizontal coordinate, the vertical coordinate, and the category label.

[0096] The filtering module is run on the initial set of candidate symbol positions to perform spatial nonmaximum suppression and minimum spacing constraint filtering, thereby generating a set of candidate symbol positions.

[0097] In the symbol location inference model, the classification branch outputs the category probability distribution of the candidate symbol location set, and the regression branch outputs the location coordinate confidence distribution of the candidate symbol location set.

[0098] Add graph index information to each candidate symbol position in the candidate symbol position set. The graph index information includes the node identifier, adjacent edge identifiers, and coordinates in the graph.

[0099] Output the set of candidate symbol locations, the category probability distribution, and the confidence distribution of location coordinates.

[0100] This invention utilizes a graph encoder and an aggregation module to extract graph structure embedding representations during the generation and prediction of candidate symbol positions. Then, a candidate set is generated through a spatial decoder, and nonmaximum suppression and minimum spacing constraints are introduced. Simultaneously, the classification branch and regression branch are combined to output the category probability distribution and the position confidence distribution, respectively. Graph index information is added to the candidate symbol positions to form a unified data structure, ensuring the diversity and compactness of the candidate set, avoiding redundancy and ambiguity among candidates, and providing reliable input for subsequent screening and ranking.

[0101] In this embodiment, the calculation of the hierarchical mutual information index specifically includes:

[0102] Read the candidate symbol location set, category probability distribution, and location coordinate confidence distribution;

[0103] Based on the topological relationships in the process diagram structure and the spatial neighborhood relationships of the candidate symbol positions, the set of candidate symbol positions is divided into a global topological layer candidate set and a local candidate layer candidate set.

[0104] A joint probability distribution and marginal distribution are established between the global topology layer candidate set and the class probability distribution and the location coordinate confidence distribution; a joint probability distribution and marginal distribution are also established between the local candidate layer candidate set and the class probability distribution and the location coordinate confidence distribution.

[0105] Based on the definition of mutual information in information theory, the logarithm of the ratio of the joint probability distribution to the marginal distribution is taken and then summed with weights to form a hierarchical mutual information index. Recorded as:

[0106] ;

[0107] in, This is the global topology layer candidate set. For the local candidate layer candidate set, For a set of symbol categories, For a set of position coordinates, The position of the candidate symbol for the global topology layer. This represents the position of candidate symbols in the local candidate layer. For symbol categories, The position coordinate vector, The joint probability distribution of the global topology layer. The joint probability distribution of the local candidate layers. The edge distribution of candidate symbol positions in the global topology layer. The edge distribution of candidate symbol positions in the local candidate layer. The joint marginal distribution of category and location coordinates, and Let be the weighting coefficient, satisfying , It is the natural logarithm.

[0108] In the calculation of hierarchical mutual information index, this invention divides the global topological layer and the local candidate layer according to the topological relationship of the process diagram structure and the spatial neighborhood relationship of the candidate positions, and establishes joint distribution and marginal distribution respectively. The hierarchical index is formed by logarithmic weighted summation of mutual information, so as to characterize the information relationship between symbol position and category coordinate at different levels. This allows the sorting process to consider both global constraints and local details, effectively improving the scientificity and rationality of symbol candidate position sorting.

[0109] In this embodiment, the generation of the mutual information sorting set specifically includes:

[0110] Read the candidate symbol location set, category probability distribution, location coordinate confidence distribution, and hierarchical mutual information index;

[0111] Generate a prediction distribution for each candidate symbol location in the candidate symbol location set. The prediction distribution consists of a class probability distribution and a location coordinate confidence distribution.

[0112] Monte Carlo sampling is performed on the prediction distribution to construct a set of parameter samples. The category prediction distribution of candidate symbol positions is calculated under each parameter sample. Given the parameter samples, the batch output conditional independence assumption is adopted, and the batch joint prediction distribution is represented as a product of the prediction distributions of individual candidate symbol positions.

[0113] Based on the definition in information theory that mutual information equals the difference between entropy and conditional entropy, and combining the conditional independence assumption and the Monte Carlo approximation, the joint information gain estimate of BatchBALD is calculated as follows:

[0114] ;

[0115] In the formula, This represents the Monte Carlo estimate of the joint information gain of BatchBALD. " indicates an estimation operator, " " represents the mutual information operator, This represents a random variable composed of the classification outputs of candidate symbol positions within a batch. Indicates batch size. Indicates the model parameters inferred from the sign position. Indicates the first One parameter sample, The parameter is the sample index and its value range is... arrive , Indicates the number of parameter samples. Represents a set of symbol categories. Indicates length is A set of category sequences Represents a category sequence, The index of the candidate symbol position within the batch and the range of values ​​is arrive , Indicates in parameter samples Next The category prediction distribution of each candidate symbol position. This represents the batch joint prediction distribution obtained by averaging over the parameter sample set;

[0116] Based on the joint information gain estimation and the hierarchical mutual information index, the candidate symbol position set is combined and sorted according to the unified sorting criterion to generate a mutual information sorted set.

[0117] This invention constructs a parameter sample set through Monte Carlo sampling during the BatchBALD ranking process, and approximates the batch joint prediction distribution under the conditional independence assumption. It uses the principle that mutual information equals the difference between entropy and conditional entropy to form a joint information gain estimate, and then combines the hierarchical mutual information index to uniformly rank the candidate set, thereby achieving the optimal combination selection of candidate positions. This effectively avoids information redundancy and improves the global gain of ranking, ensuring that subsequent screening can focus on the most representative candidate symbol positions.

[0118] In this embodiment, the execution of the two-stage screening strategy specifically includes:

[0119] Perform a two-stage filtering strategy in the mutually information-ranked set;

[0120] In the first stage, the category probability distribution, location coordinate confidence distribution, and hierarchical mutual information index of the candidate symbol location set are read. A surrogate index value is calculated for each candidate symbol location. The surrogate index value is obtained by weighting the category probability distribution score, the location coordinate confidence distribution score, and the hierarchical mutual information index. Candidate symbol locations with surrogate index values ​​not less than the threshold are formed into a pre-screening set.

[0121] In the second stage, the joint information gain estimation of BatchBALD is called within the pre-screened set to calculate the ranking score for each candidate symbol position in the pre-screened set, and the ranking order is adjusted in combination with the corresponding hierarchical mutual information index to form the precise screening set.

[0122] In this invention, a two-stage screening strategy is first used to generate surrogate indicators by weighting category probability distribution, location confidence distribution and hierarchical mutual information index to quickly eliminate redundant candidates. Then, in the pre-screened set, BatchBALD is called to jointly estimate information gain and combine hierarchical mutual information to adjust the sorting to form an accurate screening set. This ensures screening efficiency while further improving the accuracy and stability of the screening results, and achieves significant compression of the candidate set size and improvement of information fidelity.

[0123] In this embodiment, the execution of the constraint projection and k-DPP sampling specifically includes:

[0124] Perform constraint projection operations in the precise selection set to adjust the positions of candidate symbols that do not meet the symbol spacing constraint, alignment constraint, and pipeline connectivity constraint into the feasible solution set;

[0125] The symbol spacing constraint is determined by calculating the sum of the squares of the differences between the x-coordinates and y-coordinates between any two candidate symbol positions. When the result is less than a preset threshold, the candidate symbol position is moved in the coordinate plane until the spacing threshold is met.

[0126] Alignment constraints are determined by detecting the angle between the center coordinates of the candidate symbol position and the pipeline direction. When the angle is not equal to zero degrees or ninety degrees, the candidate symbol position is adjusted along the nearest direction until the center coordinates are horizontally or vertically aligned with the pipeline.

[0127] Pipeline connectivity constraints are determined by measuring the Euclidean distance between the candidate symbol position and the pipeline endpoint or pipeline midpoint. When the distance is greater than the preset radius, the candidate symbol position is moved to the boundary of the circular neighborhood range.

[0128] Construct a kernel matrix in the feasible solution set. Each element in the kernel matrix is ​​obtained by the inner product of the feature vectors of the candidate symbol positions. The feature vectors include geometric position, class encoding and adjacency relation.

[0129] The k-DPP sampling algorithm is run on the kernel matrix to select a fixed number of candidate symbol positions from the feasible solution set and generate a batch query sample set.

[0130] This invention ensures that candidate positions meet geometric and process logic requirements through symbol spacing constraints, alignment constraints, and pipeline connectivity constraints during the constraint projection and k-DPP sampling process. It also constructs a kernel matrix in the feasible solution set and establishes correlations using the inner product of eigenvectors. Then, it runs k-DPP sampling to generate a batch query set that takes into account diversity. This can improve the representativeness and coverage of the sampled samples while ensuring feasibility, and provide high-quality input for manual annotation and model updates.

[0131] In this embodiment, the introduction of the memory unit and the model update specifically include:

[0132] A memory unit is introduced during the batch query sample set generation process. The memory unit stores sample identifiers, feature vectors, graph index information, selection rounds, annotation status, contribution scores, and sampling weights.

[0133] Read the selected sample records from the history, and for each record, calculate the training loss reduction, cumulative gradient path value, and time interval between selection rounds, calculate the contribution score, and write it into the memory unit;

[0134] The contribution score is mapped to the sampling weight adjustment amount, and the candidate symbol position is matched with the memory unit entry and the sampling weight is updated according to the index in the batch query sample set.

[0135] Based on the updated sampling weights, resampling is performed within the batch query sample set to generate a labeled list;

[0136] Submit the annotation list for manual annotation, and collect the annotation results of symbol categories and location coordinates;

[0137] Based on the graph index information, a one-to-one correspondence is established between the annotation results and the candidate symbol position set, training sample entries are generated and written into the training dataset;

[0138] Based on the training dataset, a parameter update process is performed to update the symbol position inference model parameters and synchronously update the label status and contribution score in the memory unit.

[0139] Output the potential location coordinates of P&ID symbols and the inference results of symbol categories, and save the location confidence records.

[0140] This invention quantifies historical samples by recording sample identifiers, feature vectors, graph index information, selection rounds, annotation status, and contribution scores during the memory unit and model update process. It calculates the contribution based on the loss reduction and gradient path cumulative value, and then maps the contribution to the sampling weight adjustment. After resampling and manual annotation, the data is written into the training dataset and the model parameters and memory unit status are updated. This achieves dynamic adaptive closed-loop optimization, enabling the symbol position inference model to continuously improve recognition accuracy and generalization ability.

[0141] Example 1:

[0142] To verify the feasibility of this invention in practice, it was applied to the task of intelligent identification and location inference of symbols on P&ID drawings in a chemical process design department. In this scenario, engineers often need to process large-scale process drawings, with each set of equipment containing hundreds of pages of drawings. Each page contains symbols such as pumps, valves, heat exchangers, pressure gauges, and flow meters. Traditional manual review methods are time-consuming and labor-intensive, and are prone to errors due to symbol overlap, pipeline intersections, and text interference. Therefore, there is an urgent need for an intelligent method that can improve the accuracy and efficiency of identification. The existing method used for comparison is a combination detection method based on template matching and convolutional neural network classification. This method first generates candidate positions through symbol templates, then uses a convolutional neural network to output confidence scores, and finally filters them using a fixed threshold. Since it does not consider topological relationships and global information gain, this comparative method has limited performance in complex scenarios.

[0143] In application, this invention first preprocesses the P&ID drawing image, including noise removal, distortion correction, and grayscale normalization. Simultaneously, it performs word segmentation, recognition, and structured parsing on the drawing text to generate a text feature set. Then, in the vectorization step, it combines the connection set and the symbol set to form a process drawing structure, which is then input into the symbol position inference model. The model generates a graph structure embedding representation through a graph encoder and a cross-edge aggregation module. The spatial decoder then outputs an initial set of candidate symbol positions containing coordinates and category labels. Subsequently, spatial nonmaximum suppression and minimum spacing constraints are performed in the filtering module to obtain a set of candidate symbol positions. For each candidate symbol position, a category probability distribution and coordinate confidence distribution are output. Compared with the comparison method, this process can generate a more comprehensive candidate set and reduce the problem of missed detections due to insufficient local features.

[0144] In the candidate set screening stage, this invention introduces a hierarchical mutual information index to model and fuse the global topology layer candidate set and the local candidate layer candidate set respectively. Then, guided by this index, the BatchBALD algorithm is run to perform joint information gain ranking on the candidate set, ensuring that the ranking result simultaneously reflects the correlation between candidate positions and the overall information coverage. Based on this, a two-stage screening strategy is further adopted. In the first stage, a surrogate index is formed using a weighted combination of class probability, position confidence, and hierarchical mutual information, thereby quickly eliminating redundant candidate symbol positions and forming a pre-screened set. In the second stage, further information gain is used to... The atchBALD algorithm, combined with information gain, recalculates the ranking score and updates the ranking based on hierarchical mutual information index, resulting in a smaller but more accurate precise selection set. In an experiment, for a P&ID drawing containing 112 pages and 950 symbols, the traditional comparison method output 870 candidate positions, including 132 incorrect candidates and 88 missed candidates. The final selection set overlapped with the true symbol positions by 79%. In contrast, the present invention output 940 candidates on the same drawing, including 37 incorrect candidates and 28 missed candidates. The precise selection set overlapped with the true symbol positions by 94%.

[0145] In the constraint projection and sampling stage, this invention applies constraints on symbol spacing, alignment, and pipeline connectivity to the precise screening set to ensure the rationality of candidate positions in terms of geometric relationships and process logic. For candidate positions with insufficient symbol spacing, translation adjustment is performed; for candidate positions where the angle between the symbol center and the pipeline direction does not meet the horizontal or vertical relationship, direction adjustment is performed; and for candidate positions whose distance from the pipeline endpoint or midpoint exceeds the neighborhood radius, movement correction is performed. Finally, a set of 195 feasible solutions that meet the constraints is formed. Based on this, a kernel matrix is ​​constructed and k-DPP sampling is run to obtain a batch query sample set containing 50 candidate symbol positions. Subsequently, a memory unit is introduced to statistically analyze the contribution of historically selected samples, and manual annotation feedback is used to update the model parameters. After two rounds of iteration, the overall accuracy is improved to 96%, while the accuracy of the comparison method in the same scenario is only 81%. The efficiency advantage of this invention is also very obvious, with an average processing time of 4.1 seconds per page of drawing, while the comparison method takes 11.8 seconds.

[0146] To more intuitively demonstrate the comparison effect, the experimental results of the same batch of P&ID drawings are shown in Table 1:

[0147] Table 1 Performance Comparison of P&ID Symbol Latent Location Inference Methods and Comparison Methods

[0148]

[0149] The results in the table show that the comparative method is prone to misjudgment and missed detection in complex process drawing environments, the screening set is too large, and the symbol position is inconsistent with the process logic. In contrast, the present invention has achieved systematic optimization in candidate generation, information gain ranking, screening strategy and constraint projection, which ensures the accuracy and rationality of symbol position inference. Furthermore, the model is continuously improved through memory units and closed-loop updates of manual annotation. Therefore, it is significantly better than the comparative method in terms of symbol position recognition accuracy, process logic consistency and processing efficiency, and can meet the application requirements of intelligent processing of large-scale P&ID drawings in complex industrial scenarios.

[0150] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A machine learning-based method for inferring the latent location of P&ID symbols, characterized in that, Includes the following steps: Acquire P&ID drawing images and text data, perform preprocessing and vectorization, and construct the process diagram structure; In the process diagram structure, a set of candidate symbol positions is generated by the symbol position inference model, and a prediction distribution and confidence level are output for each candidate symbol position; The candidate symbol location set is divided into a global topological layer candidate set and a local candidate layer candidate set. The mutual information of each set is calculated and then fused into a hierarchical mutual information index. The BatchBALD algorithm is run based on the hierarchical mutual information index to generate a mutual information sorted set with joint information gain sorting. A two-stage screening strategy is adopted for the mutual information ranking set. In the first stage, a pre-screening set is formed based on the surrogate index. The surrogate index value is obtained by weighting the category probability distribution score, the location coordinate confidence distribution score and the hierarchical mutual information index. In the second stage, the mutual information ranking is updated on the pre-screening set to obtain the precise screening set. Perform constraint projection operations on the precise filter set to map the positions of candidate symbols that do not meet the constraints of symbol spacing, alignment and pipeline connectivity to the feasible solution set, and use k-DPP sampling on the feasible solution set to generate a batch query sample set. In the process of generating a batch query sample set, a memory unit is introduced to adjust the candidate weights based on the contribution of historically selected samples, and the symbol position inference model is updated in combination with the results of manual annotation, outputting the potential position coordinates of P&ID symbols and the inference results of symbol categories.

2. The method for inferring the latent location of P&ID symbols based on machine learning according to claim 1, characterized in that, The construction of the process diagram structure specifically includes: Acquire P&ID drawing images and text data, perform noise reduction, distortion correction and grayscale normalization on the drawing images, and generate preprocessed images; The text data is segmented, character recognized, and structured parsed to generate a set of text features; Edge detection and line extraction are performed on the preprocessed image to generate a set of connected lines; Symbol detection is performed on the preprocessed image to generate a symbol set, where each symbol contains its center coordinates; The connection set and symbol set are vectorized and combined with the text feature set to generate a process diagram structure. The process diagram structure contains a node set and an edge set. The node set corresponds to the symbol set and the text feature set, and the edge set corresponds to the connection set.

3. The method for inferring the latent location of P&ID symbols based on machine learning according to claim 1, characterized in that, The generation and prediction distribution of the candidate symbol location set and the confidence output specifically include: Load the symbol position inference model into the process diagram structure, and input the node set and edge set into the symbol position inference model; In the symbol position inference model, the features of the node set and edge set are encoded by a graph encoder to generate node feature vectors and edge feature vectors; In the symbol position inference model, node feature vectors and edge feature vectors are fused by an aggregation module to generate a graph structure embedding representation; In the symbol location inference model, a preliminary set of candidate symbol locations is generated based on graph structure embedding representation using a spatial decoder. Each candidate symbol location in the preliminary set contains the horizontal coordinate, the vertical coordinate, and the category label. The filtering module is run on the initial set of candidate symbol positions to perform spatial nonmaximum suppression and minimum spacing constraint filtering, thereby generating a set of candidate symbol positions. In the symbol location inference model, the classification branch outputs the category probability distribution of the candidate symbol location set, and the regression branch outputs the location coordinate confidence distribution of the candidate symbol location set. Add graph index information to each candidate symbol position in the candidate symbol position set. The graph index information includes the node identifier, adjacent edge identifier, and coordinates in the graph.

4. The method for inferring the latent location of P&ID symbols based on machine learning according to claim 1, characterized in that, The calculation of the hierarchical mutual information index specifically includes: Read the candidate symbol location set, category probability distribution, and location coordinate confidence distribution; Based on the topological relationships in the process diagram structure and the spatial neighborhood relationships of the candidate symbol positions, the set of candidate symbol positions is divided into a global topological layer candidate set and a local candidate layer candidate set. A joint probability distribution and marginal distribution are established between the global topology layer candidate set and the class probability distribution and the location coordinate confidence distribution; a joint probability distribution and marginal distribution are also established between the local candidate layer candidate set and the class probability distribution and the location coordinate confidence distribution. According to the definition of mutual information in information theory, the logarithm of the ratio of the joint probability distribution to the marginal distribution is taken and then summed with weights to form a hierarchical mutual information index.

5. The method for inferring the latent location of P&ID symbols based on machine learning according to claim 1, characterized in that, The generation of the mutual information sorted set specifically includes: Read the candidate symbol location set, category probability distribution, location coordinate confidence distribution, and hierarchical mutual information index; Generate a prediction distribution for each candidate symbol location in the candidate symbol location set. The prediction distribution consists of a class probability distribution and a location coordinate confidence distribution. Monte Carlo sampling is performed on the prediction distribution to construct a set of parameter samples. The category prediction distribution of candidate symbol positions is calculated under each parameter sample. Given the parameter samples, the batch output conditional independence assumption is adopted, and the batch joint prediction distribution is represented as a product of the prediction distributions of individual candidate symbol positions. In the process of joint information gain estimation, according to the definition in information theory that mutual information equals the difference between entropy and conditional entropy, the joint information gain estimate is obtained by comparing the entropy value of the candidate set under the overall prediction and the average entropy value under the parameter sample condition. Based on the joint information gain estimate and the hierarchical mutual information index, the candidate symbol position set is combined and sorted according to the unified sorting criterion to generate a mutual information sorted set.

6. The method for inferring the latent location of P&ID symbols based on machine learning according to claim 1, characterized in that, The execution of the two-stage screening strategy specifically includes: Perform a two-stage filtering strategy in the mutually information-ranked set; In the first stage, the category probability distribution, location coordinate confidence distribution, and hierarchical mutual information index of the candidate symbol location set are read. A surrogate index value is calculated for each candidate symbol location. The surrogate index value is obtained by weighting the category probability distribution score, the location coordinate confidence distribution score, and the hierarchical mutual information index. Candidate symbol locations with surrogate index values ​​not less than the threshold are formed into a pre-screening set. In the second stage, the joint information gain estimation of BatchBALD is called within the pre-screened set to calculate the ranking score for each candidate symbol position in the pre-screened set, and the ranking order is adjusted in combination with the corresponding hierarchical mutual information index to form the precise screening set.

7. The method for inferring the latent location of P&ID symbols based on machine learning according to claim 1, characterized in that, The execution of the constraint projection and k-DPP sampling specifically includes: Perform constraint projection operations in the precise selection set to adjust the positions of candidate symbols that do not meet the symbol spacing constraint, alignment constraint, and pipeline connectivity constraint into the feasible solution set; The symbol spacing constraint is determined by calculating the sum of the squares of the differences between the x-coordinates and y-coordinates of any two candidate symbol positions. When the result is less than a preset threshold, the candidate symbol position is moved in the coordinate plane until the spacing threshold is met. The alignment constraint is determined by detecting the angle between the center coordinates of the candidate symbol position and the pipeline direction. When the angle is not equal to zero degrees or ninety degrees, the candidate symbol position is adjusted along the nearest direction until the center coordinates are horizontally or vertically aligned with the pipeline. Pipeline connectivity constraints are determined by measuring the Euclidean distance between the candidate symbol position and the pipeline endpoint or pipeline midpoint. When the distance is greater than the preset radius, the candidate symbol position is moved to the boundary of the circular neighborhood range. Construct a kernel matrix in the feasible solution set. Each element in the kernel matrix is ​​obtained by the inner product of the feature vectors of the candidate symbol positions. The feature vectors include geometric position, class encoding and adjacency relation. The k-DPP sampling algorithm is run on the kernel matrix to select a fixed number of candidate symbol positions from the feasible solution set and generate a batch query sample set.

8. The method for inferring the latent location of P&ID symbols based on machine learning according to claim 1, characterized in that, The introduction of the memory unit and the model update specifically include: A memory unit is introduced during the batch query sample set generation process. The memory unit stores sample identifiers, feature vectors, graph index information, selection rounds, annotation status, contribution scores, and sampling weights. Read the selected sample records from the history, and for each record, calculate the training loss reduction, cumulative gradient path value, and time interval between selection rounds, calculate the contribution score, and write it into the memory unit; The contribution score is mapped to the sampling weight adjustment amount, and the candidate symbol position is matched with the memory unit entry and the sampling weight is updated according to the index in the batch query sample set. Based on the updated sampling weights, resampling is performed within the batch query sample set to generate a labeled list; Submit the annotation list for manual annotation, and collect the annotation results of symbol categories and location coordinates; Based on the graph index information, a one-to-one correspondence is established between the annotation results and the candidate symbol position set, training sample entries are generated and written into the training dataset; Based on the training dataset, a parameter update process is performed to update the symbol position inference model parameters and synchronously update the label status and contribution score in the memory unit. Output the potential location coordinates of P&ID symbols and the inference results of symbol categories, and save the location confidence records.

Images