An image recognition-based full-process intelligent matching method and system

Abstract

The application provides a kind of full-process intelligent point-to-point method and system based on image recognition, relating to power system technical field.The application realizes graph-text decoupling through double-branch convolution network and mutual exclusivity loss function, solves the problem of text and graphic element confusion;Utilize transfer learning and hierarchical unfreezing strategy to optimize the character recognition model, effectively deal with complex working conditions;Improved YOLOv5 combined with block detection, edge expansion and soft non-maximum suppression algorithm is used to improve the detection accuracy of small target graphic elements;Finally, the image recognition result is cross-verified with the database point table, the pre-acquisition signal and the alarm window information, breaking through the bottleneck of only comparing data level, solving the problem of inaccurate point-to-point result.The application integrates point-to-point process through joint recognition of text and image, realizes full-automatic analysis and verification of graph-text information in power primary wiring diagram, and improves the accuracy and safety of intelligent point-to-point.

Description

Technical Field

[0001] This invention relates to the field of power system technology, and in particular to a full-process intelligent point-to-point method and system based on image recognition. Background Technology

[0002] With the continuous improvement of power system automation, "point-to-point joint commissioning" between the dispatch master station and the substation has become a crucial link in ensuring the consistency of information in the power monitoring system. The main content of point-to-point commissioning is to verify the consistency of telemetry (status variables) and telemonitoring (analog variables) information between the master station and the substation, including equipment status, telemetry values, and alarm information. Traditional point-to-point commissioning methods rely on manual telephone coordination to trigger signals and manual verification of alarm windows and wiring diagrams, resulting in low efficiency and lack of process traceability. Therefore, the industry has explored various automatic point-to-point commissioning technologies.

[0003] Existing automatic point-to-point matching technologies mainly fall into two categories. The first category is based on protocol parsing, such as comparing the object address and time flag bits in the IEC 104 protocol to achieve logical-level verification. This method relies solely on data matching at the communication protocol level and does not involve parsing the graphical information on the monitoring interface; therefore, it cannot detect inconsistencies between the protocol data and the graphical display. The second category is based on data verification, such as using checksum algorithms to verify the consistency of the point table, or improving acceptance efficiency through monitoring data feature classification. Other solutions use graphical file comparison or pixel-level image comparison to verify the consistency of graphical files.

[0004] However, in primary wiring diagrams of power systems, textual information such as equipment names, numbers, and telemetry values ​​are spatially tightly coupled with electrical elements (such as circuit breakers, disconnectors, and grounding switches). Text and elements often obscure each other, font formats are inconsistent, and elements exhibit multiple states (such as closed / open). Existing recognition algorithms struggle to effectively handle this text-image coupling scenario: template-matching methods are sensitive to element deformation and have low accuracy; singular value decomposition methods are limited by image segmentation quality; and while detection models based on deep learning such as Faster R-CNN offer some improvement, they are still prone to false positives and false negatives in complex scenarios with dense small targets and overlapping text and images. Furthermore, existing automatic point-to-point matching technologies are often limited to database-level point table data comparison, lacking the ability to jointly recognize text and image information in wiring diagrams. This makes it difficult to accurately determine whether the state of elements has changed correctly after telemetry changes or whether the telemetry values ​​match the interface display, resulting in insufficient accuracy in the point-to-point matching results. Summary of the Invention

[0005] This invention provides a full-process intelligent point-to-point method and system based on image recognition, which solves the technical problem of low accuracy in existing point-to-point methods, realizes intelligent point-to-point connection in power systems, and improves the accuracy and security of point-to-point connection.

[0006] In a first aspect, the present invention provides a full-process intelligent point-to-point method based on image recognition. This method includes: acquiring the primary power wiring diagram of the target power plant, as well as database point table information, pre-acquisition signals, and alarm window information from the dispatch master station; performing joint text and image recognition on the primary power wiring diagram to extract text features and graphic element features, obtaining multiple text regions and graphic element regions; performing power text recognition on the multiple text regions to obtain power text recognition results; performing graphic element status recognition on the multiple graphic element regions to obtain graphic element status recognition results; based on the power text recognition results and graphic element status recognition results, combined with database point table information, pre-acquisition signals, and alarm window information, performing multimodal cross-validation to obtain validation results; and generating an intelligent point-to-point report based on the validation results.

[0007] Secondly, embodiments of the present invention provide a full-process intelligent point-to-point system based on image recognition. The system includes an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor is used to call and run the computer program stored in the memory to perform the steps of the method as described in the first aspect and any possible implementation thereof.

[0008] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the method as described in the first aspect and any possible implementation thereof.

[0009] This invention provides a full-process intelligent point-to-point method and system based on image recognition. It achieves image-text decoupling through a dual-branch convolutional network and a mutual exclusion loss function, solving the pixel-level confusion problem caused by the spatial coupling of text and primitives. It optimizes the text recognition model using transfer learning and a hierarchical unfreezing strategy, effectively handling complex conditions such as occlusion, blurring, and inconsistent font formats. It employs an improved YOLOv5 algorithm combined with block detection, edge expansion, and soft non-maximum suppression to improve the detection accuracy of dense small target primitives. Finally, it performs multimodal cross-validation of the image recognition results with database point tables, pre-acquisition signals, and alarm window information, overcoming the bottleneck of existing technologies that are limited to data-level comparison. This achieves closed-loop verification of four-dimensional consistency (image, modality, data, and window), solving the blind spot of data accuracy but image inaccuracy during the point-to-point process. By integrating text-image joint recognition, power text recognition, and primitive status recognition technologies into the point-to-point process, this invention achieves fully automatic parsing and verification of text and image information in power primary wiring diagrams, improving the accuracy and security of intelligent point-to-point matching. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the architecture of a full-process intelligent peer-to-peer system based on image recognition provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a full-process intelligent point-to-point method based on image recognition provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an intelligent point-to-point process provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of an intelligent peer-to-peer technology framework provided by an embodiment of the present invention; Figure 5 This is a flowchart of a soft nonmaximum suppression algorithm provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a full-process intelligent point-to-point experimental system provided by an embodiment of the present invention; Figure 7 This is a schematic diagram of the accuracy iteration in a training process provided by an embodiment of the present invention; Figure 8 This is a schematic diagram of the training process loss provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0013] To improve the efficiency, accuracy, and reliability of point-to-point communication, this invention integrates advanced information technology and automated control methods, focusing on the most critical and complex functions of remote signaling and telemetry point-to-point communication. It proposes a full-process intelligent point-to-point communication technology based on image recognition and designs a full-process intelligent point-to-point communication system based on image recognition, the architecture of which is as follows: Figure 1As shown, the dispatch master station establishes a direct and efficient connection with station maintenance personnel via wireless communication through the automatic point-to-point module in Zone I. After receiving and confirming the instructions, station maintenance personnel can conduct remote signaling and testing. Upon receiving information from the station, the dispatch master station automatically compares the consistency of signals received from the point-to-point module, signals received from the monitoring backend, and signals sent from the process layer. It also uses image recognition technology to verify relevant information such as the interval diagram and main wiring diagram in the monitoring backend.

[0014] like Figure 2 As shown, the present invention provides a full-process intelligent point-to-point method based on image recognition, including steps S101-S106. S101. Obtain the primary power wiring diagram of the target plant, as well as the database point table information, pre-collected signals, and alarm window information of the dispatch master station.

[0015] It should be noted that the full-process intelligent point-to-point mode refers to a point-to-point mode where station-side maintenance personnel modify the status of primary equipment to trigger signals for verification by sending them to the master station. When the plant / station sends a point-to-point request, the automatic point-to-point module issues the point-to-point content based on the on-site application. After receiving the point-to-point request, the master station initiates the point-to-point process, analyzes the diagram and database feedback results, and identifies whether the sent data matches the data of the target measurement point to be verified. The analysis focuses on whether the wiring diagram elements of the remote signaling signals have been correctly positioned and whether the remote measurement measurements of the telemetry signals have changed correctly. If any abnormality is detected during the analysis, the system will immediately activate the emergency mechanism, promptly terminate the point-to-point process, and issue a clear and explicit prompt through the system monitoring interface. The intelligent point-to-point process is as follows: Figure 3 As shown.

[0016] When the master station sends a confidence command for a specific point number to the plant station, it simultaneously acquires image information from the target page. When the database or front-end detector detects a change in the corresponding point's value and uploads it, it acquires image information from the target page again. Based on the uploaded signal point, it determines the position, size, and offset of that signal point in the wiring diagram, thus identifying the target area. Each time a signal is triggered, an automatic reset operation is required to prevent frequent flickering of graphic elements from affecting image acquisition. The intelligent point-to-point technology framework based on image recognition is as follows: Figure 4 As shown.

[0017] S102. Perform joint text and image recognition on the primary wiring diagram of the power system, extract text features and graphic feature features, and obtain multiple text regions and graphic region regions.

[0018] As one possible implementation, step S102 can be specifically implemented as steps S1021-S1026.

[0019] S1021. Input the primary wiring diagram of the power system into the dual-branch convolutional network trained with the mutual exclusion loss function, and extract the deep shared features of the primary wiring diagram of the power system through the shared feature extractor.

[0020] S1022. Input the deep shared features into the parallel text detection branch and primitive detection branch respectively. The text detection branch generates a text probability map through the convolution weights and scalar bias of the text detection network, and the primitive detection branch generates a primitive probability map through the convolution weights and scalar bias of the primitive detection network.

[0021] S1023. Calculate the mutual exclusion loss function based on the text probability graph and the primitive probability graph.

[0022] S1024. Based on the text probability map and the primitive probability map, mutual exclusion competition suppression is performed on the text probability and primitive probability at the same pixel position to reduce the pixel value in the overlapping area, thus obtaining the processed text probability map and primitive probability map.

[0023] S1025. Threshold segmentation is performed on the processed text probability map to obtain multiple text regions, as well as the position, size, and offset of each text region.

[0024] S1026. Threshold segmentation is performed on the processed primitive probability map to obtain multiple primitive regions, as well as the position, size and offset of each primitive region.

[0025] For example, in power grid wiring diagrams, textual information such as equipment names and telemetry data are closely coupled with electrical graphic elements, and text and graphic elements may overlap. Accurate separation of text and graphic elements is a crucial foundation for high success rates in image recognition. This invention proposes a dual-channel mutual exclusion detection algorithm to achieve precise separation of text and graphic elements.

[0026] Let the input image be Text and primitive features are extracted simultaneously using a dual-branch convolutional network: (1); in, For shared feature extractors, Features of the input image For text detection network convolution weights, Scalar bias for text detection networks. For primitive detection network convolution weights, Scalar bias for primitive detection networks. For text probability graphs Let H be the primitive probability map, W be the image height, 3 be the RGB three channels, and σ be the Sigmoid function. The result is normalized to a probability map, and the expression is shown in equation (2).

[0027] (2); where z is the input variable of the Sigmoid function.

[0028] By introducing mutual exclusion between text and primitives into the loss function, the model is forced to distinguish between the two types of targets at the feature learning level, thus solving the pixel-level confusion problem. A joint optimization objective is defined, namely the mutual exclusion loss function. To suppress the overlap of text and image regions, see equation (3).

[0029] (3); In the formula, For Hadama accumulation, The penalty intensity control coefficient is used, where N is the total number of pixels. P t (i) Let Pg(i) be the text probability value at the i-th pixel position; and let Pg(i) be the primitive probability value at the i-th pixel position. Total loss function in the joint identification phase As shown in equation (4): (4); in, Let the loss function be for character recognition. This is the loss function for image recognition.

[0030] S103. Perform electricity-related text recognition on multiple text regions to obtain the electricity-related text recognition results.

[0031] As one possible implementation, step S103 can be specifically implemented as steps S1031-S1035.

[0032] S1031. Construct a dual-branch character recognition network containing convolutional layers and recurrent layers, and initialize the parameters using the pre-trained weights of a general character recognition model to obtain the initialized power character recognition model.

[0033] For example, step S1031 can be specifically implemented as steps B1-B5.

[0034] B1. For a general text recognition model trained on massive scheduling and monitoring system image templates, extract the network weight parameters of the convolutional and recurrent layers to obtain a pre-trained weight parameter set.

[0035] B2. Based on the pre-trained weight parameter set, construct a two-branch character recognition network containing convolutional layers and recurrent layers, and determine the convolutional and recurrent parts of the two-branch character recognition network.

[0036] B3. Based on the convolutional and recurrent parts, the pre-trained weight parameter set is transferred to the corresponding layer for parameter initialization, resulting in the initialized power character recognition model.

[0037] S1032. Based on the initialized power character recognition model, by freezing the parameters of the convolutional layer and the recurrent layer and training only the classification head, the classification head can be quickly adapted to the special power character set, thus obtaining a preliminary trained power character recognition model.

[0038] S1033. Based on the preliminary training of the power text recognition model, a layered unfreezing training strategy is adopted to unfreeze the network parameters layer by layer from top to bottom according to a preset period, so as to obtain a layer-by-layer optimized power text recognition model.

[0039] S1034. Based on the layer-by-layer optimization of the power text recognition model, the power text recognition model is fine-tuned by unfreezing all parameters to adapt to the special text features in the power scenario, thus obtaining the final optimized power text recognition model.

[0040] S1035. Based on the final optimized power text recognition model, perform text recognition on multiple text regions one by one to obtain the power text recognition result for each text region; the power text recognition result includes the equipment number and telemetry value.

[0041] For example, identifying textual information such as equipment numbers and telemetry data is crucial for point-to-point operations. In power grid wiring diagrams, textual information is often obscured, blurry, or inconsistent in font size and format. Traditional optical character recognition algorithms are prone to false positives and false negatives.

[0042] This invention proposes a joint detection algorithm using a dual-branch convolutional network. It pre-trains a general character recognition model using image samples from a massive scheduling and monitoring system, while retaining the network weight parameters of its convolutional and recurrent layers. The network is defined as shown in equation (5).

[0043] (5); in, As training samples, For image Visual feature sequence, Let be the hidden state output of the Bidirectional Long Short-Term Memory (BiLSTM) network at time t. CNN() is a convolutional neural network, and BiLSTM() is a bidirectional long short-term memory network. These are the parameters for the convolutional part of a pre-trained general character recognition model. For the parameters of the cyclic part of the pre-trained general character recognition model, the pre-trained weights are used as initialization parameters and transferred to the plant wiring diagram character recognition model designed in this invention, as shown in equation (6).

[0044] (6); in, These are the parameters for the convolution part of the text recognition model for plant wiring diagrams. These are the parameters for the loop portion of the plant wiring diagram text recognition model. To achieve good performance on the main wiring diagram set, a layered unfreezing training strategy is adopted, with t training rounds and one unfreezing cycle. First freeze and Only training the randomly initialized classification head parameters This allows the classification head to quickly adapt to the new character set and learn based on stable features provided by the pre-trained feature extractor. Each training iteration... After round t, the next layer is unfrozen starting from the top layer of the network, which is the set of trainable parameters at round t. As shown in equation (7).

[0045] (7); Where L is the total number of CNN and RNN layers. For the first The parameters of the layer. Finally, after all layers are unfrozen, all parameters... All components are used in the training process for overall fine-tuning, enabling the model to effectively adapt to the special text features in power scenarios and significantly improve recognition performance under complex conditions such as occlusion and blurring.

[0046] S104. Perform primitive state recognition on multiple primitive regions to obtain primitive state recognition results.

[0047] As one possible implementation, step S104 can be specifically implemented as steps S1041-S1045.

[0048] S1041. Based on an image containing multiple primitive regions, the image is segmented using a preset grid segmentation strategy to determine the coordinate range of each sub-region.

[0049] S1042. Based on the coordinate range of each sub-region, multiple extended sub-regions with the same resolution as the primitive region are obtained by expanding the boundary by pixels and filling the edge.

[0050] S1043. Based on each extended sub-region, the improved YOLOv5 target detection network is used to perform detection respectively to obtain local detection results.

[0051] In some embodiments, the improved YOLOv5 object detection network includes a CSPDarknet53 backbone network and an SPP+PANet feature fusion structure.

[0052] S1044. Based on the local detection results of each extended sub-region, perform mapping and merging to obtain the global detection set of each primitive region.

[0053] S1045. Based on the global detection set of each primitive region, a soft nonmaximum suppression algorithm is used to perform confidence attenuation processing on the overlapping bounding boxes to obtain the primitive state recognition result of each primitive region.

[0054] In some embodiments, the primitive state recognition result includes primitive type, displacement state, and location information.

[0055] For example, electrical wiring diagrams often feature a wide variety of device elements with uneven distribution, posing a challenge to element status recognition. This invention employs an improved YOLOv5 target detection algorithm, with block detection and edge expansion as its core. It utilizes an end-to-end convolutional neural network to achieve classification and real-time localization of multiple target elements, balancing detection speed and recognition accuracy, and significantly improving the efficiency of device element recognition.

[0056] The YOLOv5 backbone network uses CSPDarknet53, whose core idea is to reduce redundant gradient computation through CrossStage Partial Connections (CSP). Let the input feature map be... C represents the number of channels, which the CSP module divides into two parts. ,in, After processing by convolutional layers and The mathematical expression for the splicing is shown in equation (8). This is the first part of the feature map processed by the convolutional layer in the CSP module; Y is the second feature map that directly participates in the splicing in the CSP module; Y is the output feature map of the CSP module.

[0057] (8); where Concat(⋅) is the channel concatenation operation; Conv(⋅) is the convolution operation.

[0058] The feature fusion module combines Spatial Pyramid Pooling (SPP) and Path Aggregation Network (PANet) to achieve dynamic fusion of multi-scale features. PANet, through bidirectional path aggregation (top-down and bottom-up), achieves the fusion of shallow detail features and deep semantic features. Let the feature map of layer l be... The feature map of the (l+1)th layer is F l+1 The fusion process is shown in equation (9): F fused This is the fused feature map after aggregation via the PANet bidirectional path. Upsample(⋅) is the upsampling operation; (9); The detection head generates predicted bounding boxes using preset anchor frames, and its loss function is the localization loss. Confidence loss With classification loss It consists of three parts, and the total loss is As shown in equation (10): (10); where λbox is the localization loss weight coefficient; λ obj λ is the confidence loss weighting coefficient. Cls These are the classification loss weight coefficients.

[0059] The positioning loss takes into account the intersection-union ratio (IoU), the distance between the center points, and the aspect ratio, as shown in Equation (11).

[0060] (11); where ρ is the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box; b is the predicted bounding box; b gt α is the ground truth bounding box; c is the diagonal distance between the minimum bounding rectangle of the predicted box and the ground truth bounding box; α is the aspect ratio consistency weighting coefficient; v is the aspect ratio consistency metric parameter.

[0061] The confidence loss measures the confidence error of whether the prediction box contains the target, as shown in Equation (12).

[0062] (12); where S 2 B represents the total number of grid cells (S×S); B represents the number of anchor frames preset for each grid cell. This represents the predicted confidence level of the j-th anchor box in the i-th grid. The true confidence value of the j-th anchor box in the i-th grid.

[0063] Classification loss To address the class imbalance problem, Focal Loss is used to suppress the weights of easily classified samples, as shown in Equation (13).

[0064] (13); where K is the total number of target categories, α t p represents the weight coefficient of class t; t γ is the predicted probability of class t; γ is the focusing parameter (modulation factor) of FocalLoss.

[0065] To address the issue of missed detections in traditional single-image detection algorithms in dense, small-object scenes, this invention proposes a block-based detection and edge expansion fusion algorithm. First, the image is divided into blocks; taking a nine-grid segmentation as an example, each sub-region... The coordinates of the (i,j)th sub-region in the nine-square grid are given by equation (14).

[0066] (14); To avoid missing edge targets due to block cutting, the boundary of each sub-region is extended to generate an extended region as shown in Equation (15).

[0067] (15); in, This is the extended region after the (i,j)th subregion is expanded by the boundary. To increase the pixel count, the expanded region is edge-padded with the original image to maintain resolution consistency. For each expanded sub-region... Perform YOLOv5 detection separately to obtain local detection results. ,in For bounding box coordinates, For category labels, The confidence level is used to map the detection results of all sub-regions back to the original image coordinate system and merge them into a global detection set. x min ,y min The coordinates of the top-left corner of the sub-region (minimum x-value, minimum y-value); x max ,y max The coordinates of the lower right corner of the sub-region (maximum x-value, maximum y-value); D i,j b is the set of local detection results for the (i,j)th extended sub-region; k c represents the bounding box coordinates of the k-th detection result; k s is the category label for the k-th detection result; k Let be the confidence score of the k-th detection result. D is the global detection set (the set after mapping and merging all sub-region detection results). ∪ is the union operation.

[0068] Because block detection may result in the same target being detected repeatedly in different sub-regions, this invention employs a soft nonmaximum suppression algorithm to merge overlapping boxes. This reduces the risk of missed detections of dense targets by lowering the confidence level of overlapping boxes rather than directly removing them. The specific steps are as follows: Figure 5 As shown.

[0069] Equation (16) demonstrates Figure 5 The rules for updating confidence levels, among which, The attenuation coefficient is set to 0.5. Figure 5 The threshold τ is set to 0.5. (16). Among them, s j s represents the confidence level of the j-th bounding box after soft NMS update; i The original confidence level of the current highest confidence bounding box (the i-th one); exp(⋅) is the exponential function; IoU(m,b) j) represents the intersection-union ratio (IU) of the highest confidence bounding box m and the j-th bounding box; σ is the attenuation coefficient of soft NMS (taken as 0.5); τ is the confidence threshold of soft NMS (taken as 0.5).

[0070] S105. Based on the power text recognition results and graphic element status recognition results, combined with database point table information, front-end acquisition signals and alarm window information, multimodal cross-validation is performed to obtain the validation results.

[0071] As one possible implementation, step S105 can be specifically implemented as steps S1051-S1057.

[0072] S1051. Based on the point number information in the pre-acquisition signal, query the point table information in the database to determine the target measurement point data to be verified, and the target area of ​​the target measurement point data in the primary wiring diagram of the power system.

[0073] S1052. Based on the target area, extract telemetry values ​​from the power text recognition results to obtain the telemetry recognition results for the target area.

[0074] S1053. Based on the telemetry identification results of the target area and the telemetry standard values ​​in the database point table information, calculate the telemetry difference and determine whether it is within the preset threshold range to determine the telemetry comparison result.

[0075] S1054. Based on the target region, extract the metamorphic state of the primitives from the primitive state recognition results to obtain the primitive state recognition results of the target region.

[0076] S1055. Based on the image element status recognition results of the target area and the remote signaling status in the pre-acquired signals, compare whether the image element displacement status is consistent with the signal status, and determine the remote signaling comparison result.

[0077] S1056. Based on the pre-collected signals and alarm window information, compare whether the alarm window is correctly triggered and displays the corresponding signal, and determine the alarm window comparison result.

[0078] S1057. Based on the telemetry comparison results, remote signaling comparison results, and alarm window comparison results, the consistency of each modality is comprehensively judged to obtain the verification results.

[0079] S106. Generate an intelligent point-to-point report based on the verification results.

[0080] As one possible implementation, step S106 can be specifically implemented as steps S1061-S1064.

[0081] S1061. Based on the verification results that have passed the verification, record the point measurement point number, signal type, verification time, and comparison results of each mode to confirm the successful point recording.

[0082] S1062. Based on the verification result of failure, immediately stop the current point-to-point process and generate an exception prompt message on the system monitoring interface. The exception prompt message includes the exception type, exception location and difference details.

[0083] S1063. Based on the abnormal prompt information, trigger the automatic reset operation to restore the graphic elements to the initial state and determine the graphic element reset completion status.

[0084] S1064. Based on the verification results in a single point-to-point process, statistically calculate the remote signaling point-to-point accuracy rate, the telemetry point-to-point accuracy rate, and the average verification efficiency, and determine the intelligent statistical analysis results.

[0085] S1065. Based on the successful point-to-point records, abnormal prompts, element reset completion status, and intelligent statistical analysis results, generate an intelligent point-to-point report.

[0086] This invention provides a full-process intelligent point-to-point method based on image recognition. It achieves image-text decoupling through a dual-branch convolutional network and a mutual exclusion loss function, solving the pixel-level confusion problem caused by the spatial coupling of text and primitives. It optimizes the text recognition model using transfer learning and a hierarchical unfreezing strategy, effectively handling complex conditions such as occlusion, blurring, and inconsistent font formats. It employs an improved YOLOv5 algorithm combined with block detection, edge expansion, and soft non-maximum suppression to improve the detection accuracy of dense small target primitives. Finally, it performs multimodal cross-validation of the image recognition results with database point tables, pre-acquisition signals, and alarm window information, overcoming the bottleneck of existing technologies that are limited to data-level comparison. This achieves closed-loop verification of four-dimensional consistency (image, modality, data, and window), solving the blind spot of data correctness but image inaccuracy during the point-to-point process. By integrating text-image joint recognition, power text recognition, and primitive status recognition technologies into the point-to-point process, this invention achieves fully automatic parsing and verification of text and image information in power primary wiring diagrams, improving the accuracy and security of intelligent point-to-point matching.

[0087] Optionally, the image recognition-based intelligent point-to-point method provided in this embodiment of the invention further includes steps S201-S205 before step S104.

[0088] S201. Obtain the primary wiring diagram sample of the plant and substation collected by the dispatch and monitoring system.

[0089] S202. Use image annotation tools to annotate the circuit breakers, disconnectors, grounding switches, trolley circuit breakers and telemetry data elements in the primary wiring diagram sample of the plant and substation, and construct an element dataset.

[0090] S203. Based on the primitive dataset, statistically analyze the distribution of the number of samples of various primitives and identify primitive categories with insufficient sample numbers to obtain a set of primitive categories to be expanded.

[0091] S204. Based on the set of primitive categories to be expanded, at least one image enhancement method among sharpening, contrast adjustment, Gaussian noise, primitive stretching, primitive compression, brightness adjustment and saturation adjustment is used to expand the samples, and the expanded primitive training sample set is obtained.

[0092] S205. Based on the expanded primitive training sample set, an improved YOLOv5 object detection network is adopted, which includes the CSPDarknet53 backbone network and the SPP+PANet feature fusion structure. The improved YOLOv5 object detection network is obtained by iterative training using the SGD optimizer combined with WarmUp and cosine annealing decay strategies.

[0093] Thus, by performing sample distribution statistics and targeted image enhancement on the primitive dataset, this invention solves the problem of model overfitting caused by the imbalance of primitive sample quantity and insufficient samples of rare categories such as trolley circuit breakers in power wiring diagrams. It significantly improves the generalization recognition ability of the improved YOLOv5 object detection network for various primitive states, ensuring the accuracy and robustness of primitive state recognition results.

[0094] Optionally, the image recognition-based intelligent point-to-point method provided in this embodiment of the invention further includes steps S301-S304 before step S102.

[0095] S301. Based on the point-to-point request issued by the target plant, the point-to-point content is sent to the target plant through the automatic point-to-point module, and the point-to-point process is initiated by the scheduling master station to determine the point-to-point process start instruction.

[0096] S302. Based on the point-to-point process start command, send a confidence command for a certain point number to the target plant terminal, and at the same time collect the initial power primary wiring diagram of the target page to obtain the initial image.

[0097] S303. Based on the database or the front-end detection of the corresponding point's numerical change, the data is sent up, and the change in the target page's primary power wiring diagram is collected again to obtain the changed image.

[0098] S304. Based on the initial image and the changed image, perform joint text image recognition, power text recognition and graphic element status recognition respectively. By comparing the changes in graphic element status and telemetry values ​​between the initial image and the changed image, determine the graphic layer change recognition result.

[0099] S305. Based on the graphic layer change recognition results, and combined with the target measurement point data in the database point table information and the signal change information sent from the pre-acquisition signal, perform multimodal cross-validation of the consistency between the graphic layer and the data layer.

[0100] Thus, by triggering two time-series acquisitions and change comparisons before and after, this invention upgrades static single-frame recognition to dynamic change detection, realizing the time-series alignment verification of graphic layer changes and data layer signal transmission. It effectively eliminates pseudo-change interference such as screen refresh delay and network jitter, ensuring the consistency and traceability of signal transmission and graphic display during the point-to-point process.

[0101] For example, to verify the effectiveness of the intelligent peer-to-peer technology proposed in this invention, a multi-level collaborative hardware architecture system was constructed based on the power dispatch control center computer room, such as... Figure 6 As shown. The core equipment group of the system includes a signal verification workstation in Zone I, a signal verification server in Zone II, an image analysis server in Zone II, and a forwarding server in Zone III, as well as mobile terminals in the Internet zone. Each component achieves cross-zone data exchange through a dedicated power safety isolation device.

[0102] This invention uses 782 standard-compliant primary wiring diagrams of power plants collected from a regional dispatch and monitoring system. The graphic elements to be identified in these primary wiring diagrams are shown in Table 1. A primary wiring diagram element dataset is constructed using image annotation tools.

[0103] Table 1 Due to the limited sample data size and uneven distribution, overfitting can occur in network training. Therefore, to address the issue of insufficient sample data for the car switch, seven image enhancement techniques were used to expand the images: sharpening, contrast adjustment, adding Gaussian noise, primitive stretching, primitive compression, brightness adjustment, and saturation adjustment. These techniques simulate more possibilities for actual primitive shapes, ensuring a similar amount of data for each category and preventing the model from over-memorizing certain details from the training set samples. After data expansion, the final data parameters are shown in Table 2.

[0104] Table 2 Subsequently, this invention, based on image recognition methods, employs a CSPDarknet53 backbone network and an SPP+PANet structure. During training, the SGD optimizer is used, employing a WarmUp+cosine annealing decay strategy. The initial learning rate is set to 0.1, with the first 10 epochs serving as a warm-up step. Momentum and weight decay are set to 0.937 and 0.0005, respectively, to adjust the learning rate and optimize the training process. The model training utilizes an innovative iterative strategy, setting the number of training iterations to 200. Precision and recall during training are as follows: Figure 7 As shown, the training set loss and the validation set loss are as follows: Figure 8 As shown.

[0105] The trained detection model is primarily evaluated from two perspectives: detection precision and detection loss. Precision and recall measure the prediction performance of positive examples from different dimensions. Precision, from a subjective perspective, reflects the accuracy of correctly classifying positive examples; it represents the proportion of examples actually predicted as true positives. Recall, from an objective perspective, reflects the accuracy of correctly identifying positive examples; it represents the proportion of all true positives correctly predicted. Average precision is calculated by averaging the precision of different classes and is used to measure the accuracy of multi-class object detection.

[0106] For the commissioning of a certain substation, the intelligent point-to-point system designed in this invention was used for acceptance testing. The substation includes three voltage levels: 110kV, 35kV, and 10kV. The number of graphical elements included is shown in Table 3, and the number of remote signaling and telemetry points is shown in Tables 3 and 4.

[0107] Table 3 Number of Elements in Substation Wiring Diagram Table 4 Number of Substation Signals The station underwent point-to-point acceptance testing using manual point-to-point testing, traditional automatic point-to-point testing, and the intelligent point-to-point testing technology proposed in this invention. Traditional manual acceptance required 4 personnel at the plant / station level and 2 at the main station level; with automatic point-to-point testing, no further acceptance personnel are needed at the main station, requiring only 4 personnel at the plant / station level. The three methods—manual point-to-point testing, traditional automatic point-to-point testing based on image comparison, and the intelligent point-to-point testing technology proposed in this invention—were comprehensively evaluated based on four dimensions: point-to-point testing time, manpower required at the main station, remote signaling point-to-point testing accuracy, and telemetry point-to-point testing accuracy. The comparison results are shown in Table 5.

[0108] Table 5 Comparison of Point-to-Point Results In a manual point-to-point scenario, each person checks an average of 16.32 points per hour; in an intelligent point-to-point scenario, this increases to an average of 26.23 points per hour. Compared to manual point-to-point, the intelligent point-to-point scenario improves work efficiency by 60.72%.

[0109] In traditional manual point-to-point scenarios, the remote signaling accuracy rate is 95.7%, and the telemetry accuracy rate is 94.3%. In intelligent point-to-point scenarios, the verification of graphics and alarm windows is added, and the remote signaling accuracy rate is improved to 99.6%, and the telemetry accuracy rate is improved to 99.2%.

[0110] This invention can decouple text and graphics in power primary wiring diagrams where text labels and electrical elements are tightly coupled, solving problems such as text occlusion, blurring, and inconsistent font size and format, and achieving accurate recognition of text in power wiring diagrams. By combining the YOLOv5 algorithm with block detection, edge expansion fusion algorithm, and soft nonmaximum suppression algorithm, it achieves accurate recognition of the model and status changes of power wiring diagram elements. Based on the recognition algorithm proposed in this invention, an intelligent point-to-point system is built to automatically compare the actual values ​​on site with the values ​​displayed on the dispatch master station's screen monitoring system in real time, significantly improving the accuracy and efficiency of verification.

[0111] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0112] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention; the electronic device 400 includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401; when the processor 401 executes the computer program 403, it implements the steps in the above method embodiments; or, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the above device embodiments; For example, computer program 403 may be divided into one or more modules / units, one or more modules / units may be stored in memory 402 and executed by processor 401 to complete the present invention; one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of computer program 403 in electronic device 400.

[0113] The processor 401 may be a central processing unit, or it may be other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.; the general-purpose processor may be a microprocessor or any conventional processor, etc. The memory 402 can be an internal storage unit of the electronic device 400, such as a hard disk or RAM of the electronic device 400; the memory 402 can also be an external storage device of the electronic device 400, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc., equipped on the electronic device 400; furthermore, the memory 402 can include both internal storage units and external storage devices of the electronic device 400. The memory 402 is used to store computer programs and other programs and data required by the terminal; the memory 402 can also be used to temporarily store data that has been output or will be output.

[0114] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A full-process intelligent point-to-point method based on image recognition, characterized in that, include: Obtain the primary power wiring diagram of the target plant, as well as the database point table information, front-end acquisition signals, and alarm window information on the dispatch master station side; The power primary wiring diagram is subjected to joint text and image recognition to extract text features and graphic feature features, resulting in multiple text regions and graphic region regions. Electricity-related text recognition was performed on multiple text regions to obtain the electricity-related text recognition results; Perform primitive state recognition on multiple primitive regions to obtain primitive state recognition results; Based on the power text recognition results and graphic element status recognition results, combined with the database point table information, front-end acquisition signals and alarm window information, multimodal cross-validation is performed to obtain the validation results; Based on the verification results, an intelligent peer-to-peer report is generated.

2. The image recognition-based end-to-end intelligent point-to-point method according to claim 1, characterized in that, The method involves performing joint text and image recognition on the primary wiring diagram of the power system, extracting text features and graphic feature features to obtain multiple text regions and graphic feature regions, including: The power primary wiring diagram is input into a two-branch convolutional network trained with a mutual exclusion loss function, and deep shared features of the power primary wiring diagram are extracted through a shared feature extractor. The deep shared features are input into the parallel text detection branch and primitive detection branch, respectively. The text detection branch generates a text probability map through the convolutional weights and scalar bias of the text detection network, and the primitive detection branch generates a primitive probability map through the convolutional weights and scalar bias of the primitive detection network. Based on the text probability graph and the primitive probability graph, calculate the mutual exclusion loss function; Based on the text probability map and primitive probability map, mutual exclusion competition suppression is performed on the text probability and primitive probability at the same pixel position to reduce the pixel value in the overlapping area, thus obtaining the processed text probability map and primitive probability map. The processed text probability map is thresholded to obtain multiple text regions, as well as the position, size, and offset of each text region; The processed primitive probability map is segmented by thresholding to obtain multiple primitive regions, as well as the position, size, and offset of each primitive region.

3. The intelligent point-to-point method based on image recognition according to claim 1, characterized in that, The process of performing electricity-related text recognition on multiple text regions to obtain electricity-related text recognition results includes: A two-branch text recognition network containing convolutional and recurrent layers is constructed. The parameters are initialized using the pre-trained weights of a general text recognition model to obtain the initialized power text recognition model. Based on the initialized power character recognition model, by freezing the parameters of the convolutional and recurrent layers and training only the classification head, the classification head can be quickly adapted to the special power character set, thus obtaining a preliminary trained power character recognition model. Based on the pre-trained power text recognition model, a layered unfreezing training strategy is adopted to unfreeze the network parameters layer by layer from top to bottom according to a preset period, so as to obtain a layer-by-layer optimized power text recognition model. Based on the layer-by-layer optimized power text recognition model, the model is fine-tuned by unfreezing all parameters to adapt to the special text features in the power scenario, thus obtaining the final optimized power text recognition model. Based on the final optimized power text recognition model, text recognition is performed on multiple text regions one by one to obtain the power text recognition result for each text region; the power text recognition result includes the equipment number and telemetry value.

4. The image recognition-based end-to-end intelligent point-to-point method according to claim 3, characterized in that, The construction of a dual-branch text recognition network containing convolutional and recurrent layers, and the parameter initialization using pre-trained weights of a general text recognition model, yields an initialized power text recognition model, including: For a general text recognition model trained on massive scheduling and monitoring system image templates, the network weight parameters of the convolutional and recurrent layers are extracted to obtain a set of pre-trained weight parameters. Based on the pre-trained weight parameter set, a two-branch character recognition network containing convolutional layers and recurrent layers is constructed, and the convolutional part and recurrent part of the two-branch character recognition network are determined. Based on the convolutional and recurrent parts, the pre-trained weight parameter set is transferred to the corresponding layer for parameter initialization, resulting in the initialized power character recognition model.

5. The intelligent point-to-point method based on image recognition according to claim 1, characterized in that, The step of performing primitive state recognition on multiple primitive regions to obtain primitive state recognition results includes: Based on an image containing multiple primitive regions, a preset grid segmentation strategy is used to segment the image and determine the coordinate range of each sub-region. Based on the coordinate range of each sub-region, multiple extended sub-regions with the same resolution as the primitive region are obtained by expanding the boundary by pixels and filling the edge. Based on each extended sub-region, an improved YOLOv5 object detection network is used to perform detection respectively to obtain local detection results; the improved YOLOv5 object detection network includes a CSPDarknet53 backbone network and an SPP+PANet feature fusion structure. Based on the local detection results of each extended sub-region, mapping and merging are performed to obtain the global detection set of each primitive region; Based on the global detection set of each primitive region, a soft nonmaximum suppression algorithm is used to perform confidence attenuation processing on the overlapping bounding boxes to obtain the primitive state recognition result of each primitive region; the primitive state recognition result includes primitive type, displacement state and position information.

6. The intelligent point-to-point method based on image recognition according to claim 1, characterized in that, Based on the power text recognition results and graphic element status recognition results, combined with the database point table information, pre-collected signals, and alarm window information, multimodal cross-validation is performed to obtain the validation results, including: Based on the point number information in the pre-acquisition signal, query the point table information in the database to determine the target measurement point data to be verified, as well as the target area of ​​the target measurement point data in the primary wiring diagram of the power system. Based on the target area, telemetry values ​​are extracted from the power text recognition results to obtain the telemetry recognition results for the target area; Based on the telemetry identification results of the target area and the telemetry standard values ​​in the database point table information, the telemetry difference is calculated and it is determined whether it is within the preset threshold range to determine the telemetry comparison result. Based on the target region, the primitive displacement state is extracted from the primitive state recognition result to obtain the primitive state recognition result of the target region. Based on the image element state recognition results of the target area and the remote signaling state in the pre-acquired signal, compare whether the image element displacement state is consistent with the signal state to determine the remote signaling comparison result. Based on the aforementioned pre-collected signals and alarm window information, compare whether the alarm window is correctly triggered and displays the corresponding signal to determine the alarm window comparison result; Based on the telemetry comparison results, remote signaling comparison results, and alarm window comparison results, the consistency of each modality is comprehensively judged to obtain the verification results.

7. The intelligent point-to-point method based on image recognition according to claim 1, characterized in that, The generation of a smart peer-to-peer report based on the verification result includes: Based on the successful verification result, record the point measurement point number, signal type, verification time, and comparison results of each mode to confirm the successful point recording; Based on the verification result of failure, the current point-to-point process is immediately terminated and an abnormal prompt message is generated on the system monitoring interface. The abnormal prompt message includes the abnormal type, abnormal location and difference details. Based on the aforementioned error message, an automatic reset operation is triggered to restore the graphic elements to their initial state, and the graphic element reset completion status is confirmed. Based on the verification results in a single point-to-point process, the accuracy rate of remote information point-to-point, the accuracy rate of remote telemetry point-to-point, and the average verification efficiency are statistically calculated to determine the intelligent statistical analysis results. Based on the successful point-to-point records, error message information, element reset completion status, and intelligent statistical analysis results, an intelligent point-to-point report is generated.

8. The end-to-end intelligent point-to-point method based on image recognition according to any one of claims 1 to 7, characterized in that, Before performing primitive state recognition on multiple primitive regions to obtain the primitive state recognition result, the method further includes: Obtain samples of primary wiring diagrams for power plants and substations collected by the dispatch and monitoring system; Image annotation tools were used to annotate circuit breakers, disconnect switches, grounding switches, trolley circuit breakers, and telemetry data elements in the primary wiring diagram samples of the plant, and an element dataset was constructed. Based on the primitive dataset, the distribution of the number of samples of various primitives is statistically analyzed and primitive categories with insufficient sample numbers are identified to obtain a set of primitive categories to be expanded. Based on the set of primitive categories to be expanded, at least one image enhancement method among sharpening, contrast adjustment, Gaussian noise, primitive stretching, primitive compression, brightness adjustment and saturation adjustment is used to expand the samples, and the expanded primitive training sample set is obtained. Based on the expanded primitive training sample set, an improved YOLOv5 object detection network is adopted, which includes a CSPDarknet53 backbone network and an SPP+PANet feature fusion structure. The improved YOLOv5 object detection network is obtained by iterative training using an SGD optimizer combined with WarmUp and cosine annealing decay strategies.

9. The intelligent point-to-point method based on image recognition according to any one of claims 1 to 7, characterized in that, Before performing text-image joint recognition on the primary wiring diagram to extract text features and graphic feature features, and obtaining multiple text regions and graphic feature regions, the method further includes: Based on the point-to-point request issued by the target plant, the point-to-point content is sent to the target plant through the automatic point-to-point module, and the point-to-point process is initiated by the dispatch master station to determine the point-to-point process start command. Based on the point-to-point process start command, a confidence command for a certain point number is sent to the target plant terminal, and the initial power primary wiring diagram of the target page is collected to obtain the initial image. Based on the database or the front-end detection of the corresponding point, the numerical change is sent up, and the change of the target page is collected again to obtain the power primary wiring diagram after the change, and the changed image is obtained. Based on the initial image and the changed image, text image joint recognition, power text recognition and graphic element state recognition are performed respectively. By comparing the changes in graphic element state and telemetry values ​​between the initial image and the changed image, the graphic layer change recognition result is determined. Based on the graphic layer change recognition results, and combined with the target measurement point data in the database point table information and the signal change information sent from the pre-acquisition signal, a multimodal cross-validation of the consistency between the graphic layer and the data layer is performed.

10. A full-process intelligent peer-to-peer system based on image recognition, characterized in that, The system includes an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor is configured to invoke and run the computer program stored in the memory to perform the method as described in any one of claims 1 to 9.

Images