Remote control result confirmation method and system based on video detection and character recognition

Abstract

This invention provides a method and system for confirming remote control results based on video detection and text recognition. The method includes: configuring the association information between switchgear and corresponding cameras according to project site requirements; acquiring real-time video streams via the RTSP protocol of the corresponding cameras; performing frame analysis on the real-time video stream and calling a text detection model to locate text regions in the current frame; calling a text recognition model on the located text regions; comparing and verifying the recognized text content with the unique identifier information of the target switchgear; after successful location, continuously analyzing the video stream and calling an indicator light detection model trained based on the YOLOv5 architecture to detect the status of indicator lights on the switchgear in real time; encapsulating the identified indicator light status information into result data and sending it to the backend system in real time or periodically for operation result judgment. This invention eliminates the reliance on manual on-site verification, thereby achieving truly reliable remote control and status confirmation.

Description

Technical Field

[0001] This invention relates to the field of power system automation technology, specifically to a method and system for confirming remote control results based on video detection and text recognition. Background Technology

[0002] In modern power systems, switchgear, as a critical electrical device, directly affects the safe and stable operation of the power grid through the opening and closing status of its internal circuit breakers. Currently, for widely used vacuum-sealed circuit breakers, the industry commonly uses auxiliary switches mechanically linked to the circuit breaker to obtain status signals. These signals are typically associated with indicator lights on the switchgear panel and ultimately uploaded to the back-end monitoring system via remote signaling.

[0003] The specific remote control process is as follows: The back-end staff sends a remote control command (opening or closing) to the intermediate device (such as a monitoring and control protection device). After the intermediate device executes the operation, it reads the status of the auxiliary switch and converts it into a remote signaling value and returns it to the back-end. The back-end system determines whether the operation is successful based on this remote signaling value.

[0004] However, the aforementioned existing technical solutions have the following inherent drawbacks: Reliability risks in signal transmission: The entire control and feedback loop depends on the normal operation of intermediate equipment. If this intermediate equipment experiences hardware failure or software malfunction, it may return a false signaling value indicating successful operation even when no actual operation was performed. This could lead to serious misjudgments by back-end personnel and pose a significant threat to the safe operation of the power system.

[0005] Lack of intuitive final verification: When the back-end has doubts about the remote signaling value, or needs to perform high-reliability operation confirmation, the only means is to send professional personnel to the site to visually check the status of the indicator lights on the switch cabinet. This not only greatly increases the labor and time costs, but also greatly reduces the convenience and timeliness of remote control, which goes against the original intention of automated operation and maintenance. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this invention is to provide a remote control result confirmation method based on video detection and text recognition, which aims to solve the problem of unreliable remote signaling values ​​caused by intermediate equipment failure and eliminate the reliance on manual on-site verification, thereby achieving truly reliable remote control and status confirmation.

[0007] The present invention achieves the above objectives through the following technical solutions: A method for confirming remote control results based on video detection and text recognition includes: While sending remote control commands to the execution device for the target switch cabinet, the back-end system also sends a pan-tilt control command to the corresponding camera, causing the camera to rotate to a designated preset position to clearly capture the switch cabinet panel, and sends an intelligent identification request containing the unique identifier information of the target switch cabinet to the intelligent identification device. The intelligent identification device obtains real-time video streams through the RTSP protocol of the corresponding camera by configuring the switch cabinet and corresponding camera according to the needs of the project site. Perform frame analysis on the real-time video stream and call the text detection model to locate the text region in the current frame; The text recognition model is invoked to identify the text content of the located text region; The identified text content is compared and verified with the unique identifier of the target switch cabinet in the background request. If the comparison is consistent, it is determined that the camera is pointing at the target switch cabinet. After successful localization, the video stream is continuously analyzed, and the indicator light detection model trained based on the YOLOv5 architecture is called to detect the status of the indicator lights on the counter in real time. The identified indicator light status information and the video footage at the time of identification are packaged into result data and sent to the backend system in real time or periodically for operation result judgment.

[0008] According to the present invention, a remote control result confirmation method based on video detection and text recognition is provided. The text detection model adopts a three-stage detection model architecture of PP-LCNetV3 backbone network + DBFPN + PFHead neck + DBHead detection head. The PP-LCNetV3 backbone network is used to extract multi-scale features of the image. The DBPPN feature pyramid and PFHead parallel branch fusion structure are used to enhance the feature fusion capability of small-sized, slanted or curved text. The DBHead detection head is used to output text probability map, threshold map and binary map to achieve accurate segmentation of text region.

[0009] According to the present invention, a remote control result confirmation method based on video detection and text recognition is provided. The text recognition model adopts a text sequence recognition architecture of SVTR backbone network + Lite-Neck + MultiHead. The SVTR backbone network segments the text image into character components based on the static visual Transformer framework and extracts the local and global features of the text image. The Lite-Neck enhances the interaction of sequence features through the SVTR global mixing block. The MultiHead includes a CTChead branch for regular text recognition and an NRTRHead branch for irregular text recognition. Finally, the MultiHead outputs the character sequence.

[0010] According to the present invention, a remote control result confirmation method based on video detection and text recognition is provided. The indicator light detection model is a special recognition model for traffic lights trained on a proprietary dataset based on the YOLOv5 architecture and combined with the scene scene. The proprietary dataset includes at least indicator light samples under different lighting conditions and different shooting angles in the scene scene.

[0011] According to the remote control result confirmation method based on video detection and text recognition provided by the present invention, the intelligent recognition device pre-stores an association mapping table between the unique identifier of the switch cabinet configured on the project site and the RTSP address of the camera, that is, the association information between the switch cabinet and the corresponding camera. When a smart identification request containing the unique identifier of the target switch cabinet is received from the backend system, the association mapping table is queried based on the unique identifier of the target switch cabinet to obtain the corresponding camera RTSP address; The intelligent recognition device initiates a real-time video stream connection request to the camera corresponding to the RTSP address via the RTSP protocol. After the connection is established, it continuously receives and parses the real-time video stream data sent by the camera.

[0012] According to the remote control result confirmation method based on video detection and text recognition provided by the present invention, the construction of a three-stage detection model architecture includes: The PP-LCNetV3 backbone network is constructed by adopting a lightweight convolutional neural network structure. Multiple PP-LCNet modules containing depthwise separable convolution and channel shuffling operations are stacked to extract multi-scale features of the input image layer by layer. A global average pooling layer is introduced at the end of the PP-LCNet module to generate a set of feature maps containing different receptive fields. The DBFPN feature pyramid is constructed by taking the multi-scale feature maps output by the PP-LCNetV3 backbone network as input and constructing the feature pyramid through a bidirectional feature fusion path from top to bottom and bottom to top. Deformable convolutions are inserted in each bidirectional feature fusion path. The output is an enhanced multi-level feature map, with each feature map containing semantic information and spatial details at different scales. The PFHead parallel branch fusion structure is constructed, including the design of a dual-branch parallel structure. Branch 1 is a spatial attention branch, which uses a nonlocal network to capture long-range dependencies and enhance the spatial continuity of text regions. Branch 2 is a channel attention branch, which dynamically adjusts the feature channel weights through the SE module to highlight text-related features. The outputs of the two branches are fused by adding them element by element to generate a fused feature map. Constructing the DBHead detection head involves generating three outputs in parallel based on the fused feature maps: Text probabilistic graph: Generates the probability value of each pixel belonging to a text region through a 1×1 convolutional layer; Thresholding map: An adaptive threshold for each pixel is generated through a 1×1 convolutional layer; Binary image: Combining the text probability map and the threshold map, the final binarized result is generated through a differentiable binarization operation.

[0013] According to the remote control result confirmation method based on video detection and text recognition provided by the present invention, the construction of the text sequence recognition architecture includes: The SVTR backbone network is constructed by dividing the input text image into non-overlapping character components of a fixed size, with each component serving as the smallest processing unit of the Transformer module; grouping adjacent character components for computation using a local attention module to capture the spatial relationships between components and generate local feature maps; establishing long-distance dependencies among all character components using a global attention module to fuse global semantic information; and fusing the local and global feature maps through concatenation or weighted summation to output a feature sequence containing multi-scale information. Constructing a Lite-Neck involves inserting multiple global blending blocks into the feature sequence; Constructing the MultiHead header includes: The CTCHead branch is designed by employing a connection-based temporal classification (decoder) to map feature sequences to character category probability distributions; and introducing a language model to re-score the CTC output. The NRTRHead branch is designed based on a non-autoregressive Transformer architecture, which generates character sequences through parallel decoding to improve the inference speed of irregular text; a position enhancement module is used to explicitly model character position information. The output weights of the CTCHead and NRTRHead branches are automatically adjusted based on the characteristics of the input text through a dynamic weight allocation layer. The weighted character probability distribution is used to generate the final character sequence through greedy search or bundle search.

[0014] According to the remote control result confirmation method based on video detection and text recognition provided by the present invention, multiple global mixing blocks are inserted into the feature sequence, and each global mixing block includes the following sub-modules: Channel compression module: Reduces the number of feature channels by using 1×1 convolutional layers, thereby reducing computational complexity; Lightweight attention module: It adopts a linear attention mechanism instead of standard self-attention, reducing space complexity; Feature recovery module: Recovers the original number of channels through a 1×1 convolutional layer and introduces residual connections to avoid gradient vanishing; Hierarchical interaction enhancement: A bidirectional long short-term memory network is inserted between adjacent global mixing blocks to strengthen the contextual correlation of sequence features.

[0015] According to the remote control result confirmation method based on video detection and text recognition provided by the present invention, the construction of a dedicated recognition model for traffic lights includes: For the field scenario of indicator lights in power system switchgear, raw image data containing different lighting conditions, angles, obstructions, and indicator light statuses were collected; an expanded dataset was generated through geometric transformation, color space adjustment, and simulated obstruction to improve the model's generalization ability; the indicator light areas were labeled with rectangles and category labels were defined. The input layer was adjusted by modifying the input size of the original YOLOv5 to match the resolution of the scene image; a coordinate attention module was inserted into the residual block of CSPDarknet53 to enhance the model's sensitivity to the spatial position of the indicator lights; the original 3×3 convolution was replaced with deformable convolution to adaptively adjust the receptive field to adapt to tilted or deformed indicator lights. A small target detection layer is added to the path aggregation network, and shallow features are fused with deep features through upsampling operations; Focal Loss is used instead of standard cross-entropy loss to solve the problem of imbalanced positive and negative sample ratio in indicator light samples; Based on the indicator light size distribution of the on-site dataset, the anchor frame size adapted to small targets is regenerated using the K-means clustering algorithm.

[0016] A remote control result confirmation system based on video detection and text recognition includes: The back-end system is used to send remote control commands and intelligent recognition requests, and to receive indicator light status information; The camera rotates to a designated preset position according to the PTZ control command of the back-end system to capture the switch cabinet panel; The intelligent recognition device connects to a camera to receive and process the camera's real-time video stream, including text region localization, text content recognition, indicator light status recognition, and sending the recognition results to the backend system. Among them, intelligent identification devices include: The text region localization module calls a text detection model to perform frame analysis on the real-time video stream and locate text regions. The text content recognition module calls the text recognition model to recognize the text content of the located text region; The text content comparison and verification module compares and verifies the identified text content with the unique identifier of the target switchgear in the backend request. After successful localization, the indicator light status recognition module continuously analyzes the video stream and calls the indicator light detection model trained based on the YOLOv5 architecture to detect the status of the indicator lights on the counter in real time. The result sending module encapsulates the identified indicator light status information and the video footage at the time of identification into result data, and sends it to the backend system in real time or periodically.

[0017] Therefore, compared with the prior art, the remote control result confirmation method and system based on video detection and text recognition proposed in this invention have the following beneficial effects: 1. This invention introduces video target detection and text recognition technologies to construct a visual verification system independent of traditional remote signaling feedback mechanisms. It can capture and analyze image information from switchgear panels in real time, obtaining intuitive visual evidence. This visual evidence can be cross-referenced with traditional remote signaling values, thereby greatly improving the reliability of status confirmation. Even if intermediate equipment malfunctions or software abnormalities cause false alarms in remote signaling values, the visual verification system of this invention can still provide accurate status information, effectively avoiding the risk of misjudgment and ensuring the safe and stable operation of the power system.

[0018] 2. The remote control result confirmation method of this invention completely replaces the manual on-site verification process, achieving true remote control and unattended operation. Through the collaborative work of intelligent identification devices and cameras, the system can automatically complete real-time monitoring and verification of the switchgear status without manual intervention. Staff can complete the final confirmation from the monitoring center, greatly improving operation and maintenance efficiency and reducing labor costs. At the same time, this also reduces the safety risks that may arise from manual on-site operation, improving the overall safety of the power system.

[0019] 3. This invention employs a dual verification mechanism of "target detection + text recognition," achieving intelligent and accurate verification of the camera's alignment with the target. The system automatically analyzes text information in the video stream and compares it with the target identifier in the background request, ensuring that the camera is always aligned with the correct switchgear. This not only improves the system's robustness but also enhances its adaptability to the field. Regardless of changes in the switchgear layout or the complexity of the field environment, the system can quickly and accurately complete the positioning task, providing strong support for the confirmation of remote control results.

[0020] 4. The remote control result confirmation method of this invention can detect and transmit indicator light status information in real time, allowing back-end staff to observe the changes in status after operation as if they were on-site. This enhances the transparency and controllability of the operation, enabling staff to understand the actual effect of remote control in a timely manner and make adjustments or interventions as needed. Simultaneously, the system also saves video footage of the identification moment as evidence, providing strong support for subsequent fault analysis and handling.

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0022] Figure 1 This is a flowchart of an embodiment of a remote control result confirmation method based on video detection and text recognition according to the present invention.

[0023] Figure 2 This is a schematic diagram of the text detection model in an embodiment of the remote control result confirmation method based on video detection and text recognition of the present invention.

[0024] Figure 3 This is a schematic diagram of the text recognition model in an embodiment of a remote control result confirmation method based on video detection and text recognition according to the present invention.

[0025] Figure 4 This is a schematic diagram of the indicator light detection model in an embodiment of a remote control result confirmation method based on video detection and text recognition according to the present invention.

[0026] Figure 5 This is a schematic diagram of an embodiment of a remote control result confirmation system based on video detection and text recognition according to the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] An embodiment of a remote control result confirmation method based on video detection and text recognition See Figure 1 This embodiment provides a method for confirming remote control results based on video detection and text recognition, including: While sending remote control commands to the execution device for the target switch cabinet, the back-end system also sends a pan-tilt control command to the corresponding camera, causing the camera to rotate to a specified preset position to clearly capture the switch cabinet panel, and sends an intelligent identification request to the intelligent identification device. This request contains the unique identification information of the target switch cabinet, such as the cabinet number and / or cabinet name. The intelligent identification device, based on the project site requirements, configures the association information between the switch cabinets and corresponding cameras, and obtains the real-time video stream through the RTSP protocol of the corresponding camera. Specifically, the intelligent identification device needs to pre-configure and save the association information between the switch cabinets and corresponding cameras according to the project site requirements. For example, if there are 10 switch cabinets on site, numbered AK01 to AK010, each cabinet number is associated with a corresponding camera's RTSP Uniform Resource Locator (URL). After receiving a request from the backend, the intelligent identification device obtains the cabinet number from the request information, matches it with the cabinet number in the association configuration, obtains the URL of the corresponding camera's RTSP, and then obtains the real-time video stream from that camera via RTSP.

[0030] Perform frame analysis on the real-time video stream and call the text detection model to locate the text region in the current frame; The text recognition model is invoked to identify the text content of the located text region; The identified text content is compared and verified with the unique identifier of the target switch cabinet in the backend request. If they match, it is determined that the camera is pointing at the target switch cabinet; for example, if the text content identified by the model is "AK01" and the cabinet number in the request information is also "AK01", then they are considered to match. If they match, it is determined that the camera is accurately pointing at the target switch cabinet; if they do not match or the timeout occurs, the positioning is determined to have failed. Through the dual confirmation mechanism of target detection and text recognition, the accuracy of the observed object is ensured.

[0031] After successful positioning, the video stream is continuously analyzed, and the indicator light detection model trained based on the YOLOv5 architecture is called to detect the status of the indicator lights on the counter in real time, such as the color / pattern corresponding to the opening or closing of the circuit breaker. The detected indicator light status information and the video footage at the moment of detection are encapsulated into result data and sent to the backend system in real time or periodically for operation result evaluation. Specifically, the detected indicator light status information is encapsulated into result data, and the video footage at the moment of detection is saved as a PNG image and sent to the backend system in real time or periodically for staff to judge the actual result of the remote control operation.

[0032] The intelligent recognition operation will terminate when a recognition termination command is received from the backend system, or when the recognition process continues for more than the preset time.

[0033] In this embodiment, as Figure 2As shown, the text detection model adopts a three-stage detection model architecture consisting of a PP-LCNetV3 backbone network, DBFPN, PFHead neck, and DBHead detection head. The PP-LCNetV3 backbone network is used to extract multi-scale features of the image. The DBPPN feature pyramid and PFHead parallel branch fusion structure are used to enhance the feature fusion capability of small-sized, slanted, or curved text. The DBHead detection head is used to output text probability maps, threshold maps, and binary maps to achieve accurate text region segmentation. In this embodiment, the three-stage detection model architecture is the ppcrv4_det text detection model, which can locate text regions in the current image. ppcrv4_det is the core text detection model of Baidu PaddleOCR v4.

[0034] In this embodiment, as Figure 3 As shown, the text recognition model adopts a text sequence recognition architecture consisting of an SVTR backbone network, a Lite-Neck, and a MultiHead. The SVTR backbone network segments the text image into character components based on a static visual Transformer framework, extracting local and global features. The Lite-Neck enhances sequence feature interaction through SVTR global mixing blocks. The MultiHead includes a CTCHEad branch for regular text recognition and an NRTRHead branch for irregular text recognition, ultimately outputting the character sequence. In this embodiment, the text sequence recognition architecture uses the ppcrv4_det text detection model, which can locate text regions in the current image. ppcrv4_det is the core text detection model of Baidu PaddleOCR v4.

[0035] In this embodiment, as Figure 4 As shown, the indicator light detection model is a dedicated recognition model for traffic lights trained on a proprietary dataset based on the YOLOv5 architecture and combined with the actual scene. The proprietary dataset includes indicator light samples under different lighting conditions and shooting angles in the actual scene. It is specifically trained to address the recognition difficulties of indicator lights that are "small targets with multiple lighting / angles" in order to solve the problem of insufficient generalization of general target detection models for indicator light recognition and to adapt to the needs of indicator light recognition.

[0036] In this embodiment, the intelligent identification device pre-stores a mapping table that associates the unique identifier of the switch cabinet configured on-site with the RTSP address of the camera, i.e., the association information between the switch cabinet and the corresponding camera. When a smart identification request containing the unique identifier of the target switch cabinet is received from the backend system, the association mapping table is queried based on the unique identifier of the target switch cabinet to obtain the corresponding camera RTSP address; The intelligent recognition device initiates a real-time video stream connection request to the camera corresponding to the RTSP address via the RTSP protocol. After the connection is established, it continuously receives and parses the real-time video stream data sent by the camera.

[0037] Specifically, the construction of the three-stage detection model architecture includes: The PP-LCNetV3 backbone network is constructed by adopting a lightweight convolutional neural network structure. Multiple PP-LCNet modules containing depthwise separable convolution and channel shuffle operations are stacked to extract multi-scale features of the input image layer by layer. At the end of the PP-LCNet module, a global average pooling layer is introduced to generate a set of feature maps containing different receptive fields for use in subsequent stages. The DBFPN feature pyramid is constructed by taking multi-scale feature maps output from the PP-LCNetV3 backbone network as input and building the feature pyramid through bidirectional feature fusion paths from top-down to bottom-up. Deformable convolutions are inserted into each bidirectional feature fusion path to enhance the model's ability to model the spatial deformation of small-sized, tilted, or curved text. The enhanced multi-level feature maps are output, with each layer containing semantic information and spatial details at different scales.

[0038] The PFHead parallel branch fusion structure is constructed, including the design of a dual-branch parallel structure. Branch 1 is a spatial attention branch, which uses a non-local network to capture long-distance dependencies and enhance the spatial continuity of text regions. Branch 2 is a channel attention branch, which dynamically adjusts the feature channel weights through the SE (Squeeze-and-Excitation) module to highlight text-related features. The outputs of the two branches are fused by adding them element-wise to generate a fused feature map, thereby improving the ability to represent complex text morphologies.

[0039] Constructing the DBHead detection head involves generating three outputs in parallel based on the fused feature maps: Text probabilistic graph: Generates the probability value of each pixel belonging to a text region through a 1×1 convolutional layer; Thresholding map: An adaptive threshold for each pixel is generated through a 1×1 convolutional layer for subsequent binarization processing; Binary image: Combining the text probability map and the threshold map, the final binarization result is generated through the Differentiable Binarization operation, which accurately segments the text region; The problem of imbalanced positive and negative samples is solved by jointly optimizing the Dice loss function and the balanced cross-entropy loss function.

[0040] Specifically, the construction of the text sequence recognition architecture includes: The SVTR backbone network is constructed by dividing the input text image into non-overlapping character components of a fixed size (e.g., 4×4 pixels), with each component serving as the smallest processing unit of the Transformer module. A local attention module is used to group and compute adjacent character components, capturing the spatial relationships between components and generating local feature maps. A global attention module is employed to establish long-distance dependencies among all character components, fusing global semantic information. The local and global feature maps are then fused through concatenation or weighted summation to output a feature sequence containing multi-scale information.

[0041] Constructing the Lite-Neck involves inserting multiple global blending blocks into the feature sequences, each block containing the following sub-modules: Channel compression module: Reduces the number of feature channels by using 1×1 convolutional layers, thereby reducing computational complexity; Lightweight attention module: Uses linear attention mechanisms (such as Linformer) instead of standard self-attention to reduce space complexity; Feature recovery module: Recovers the original number of channels through a 1×1 convolutional layer and introduces residual connections to avoid gradient vanishing; Hierarchical interaction enhancement: A bidirectional long short-term memory network (BiLSTM) is inserted between adjacent global hybrid blocks to strengthen the contextual correlation of sequence features.

[0042] Constructing the MultiHead header includes: The CTCHead branch is designed by using a Connectionist Temporal Classification (CTC) decoder to map feature sequences to character category probability distributions. A language model is introduced to re-score the CTC output, thereby optimizing the recognition accuracy of regular text.

[0043] The NRTRHead branch is designed based on a non-autoregressive Transformer (NRTR) architecture. It generates character sequences through parallel decoding, thereby improving the inference speed of irregular text. A Positional Enhancement Module is used to explicitly model character position information, solving the problem of position loss without an autoregressive mechanism.

[0044] The dynamic weight allocation layer automatically adjusts the output weights of the CTChead and NRTRHead branches based on the characteristics of the input text (such as regularity and length). The weighted character probability distribution is used to generate the final character sequence through greedy search or beam search.

[0045] Specifically, the construction of a dedicated recognition model for traffic lights includes: For the field scenario of indicator lights in power system switchgear, raw image data is collected, including different lighting conditions (such as strong light, weak light, and reflection), angles (front, side, and tilt), occlusion (partial occlusion and complete occlusion), and indicator light status (color / pattern corresponding to open / closed). An expanded dataset is generated through geometric transformations (rotation, scaling, and cropping), color space adjustments (brightness, contrast, and saturation perturbations), and simulated occlusion (adding random rectangular occlusion blocks) to improve the model's generalization ability. Rectangular boxes are used to label the indicator light areas, and category labels (such as "red closed" and "green open") are defined.

[0046] The input layer was adjusted by modifying the original YOLOv5 input size to match the resolution of the scene image (e.g., 640×640 pixels) to avoid loss of small target information due to size scaling; a coordinate attention module (e.g., SE module or CBAM module) was inserted into the residual block of CSPDarknet53 to enhance the model's sensitivity to the spatial position of the indicator lights; the original 3×3 convolution was replaced with deformable convolution to adaptively adjust the receptive field to adapt to tilted or deformed indicator lights.

[0047] A small target detection layer is added to the Path Aggregation Network (PANet). By upsampling, shallow and deep features are fused to improve the localization accuracy of small indicator lights.

[0048] Focal Loss is used instead of standard cross-entropy loss to solve the problem of imbalanced positive and negative sample ratio in indicator light samples; Based on the indicator light size distribution of the on-site dataset, the anchor box size (e.g., [10,10], [15,15], [20,20]) is regenerated using the K-means clustering algorithm to adapt to small targets. The Soft-NMS algorithm is introduced into the non-maximum suppression (NMS) to avoid missed detections caused by indicator lights being obscured or overlapping.

[0049] A transfer learning strategy was employed, loading weights pre-trained on the COCO dataset, freezing the backbone network parameters, and fine-tuning only the neck network and detection head. During training, the input image size was randomly scaled (e.g., [640, 704, 768] pixels) to improve the model's robustness to scale changes. mAP (mean Average Precision) @ 0.5: 0.95 was used as the primary evaluation metric, and small targets (indicator area ≤ original) were separately evaluated. Figure 3 The AP value (%) is used to ensure the model's performance in complex scenarios.

[0050] In practical applications, this embodiment takes the closing operation of the "No. 1 incoming line cabinet" (cabinet number: A-01) in the power system as an example, but the application scope of the present invention is not limited to this.

[0051] High-definition cameras are installed on-site at the switchgear to ensure that each switchgear has a corresponding camera that can clearly capture its panel information, including indicator lights and cabinet number markings.

[0052] Deploy intelligent recognition equipment, which has functions such as video stream processing, text detection and recognition, and target status analysis, and is connected to the background monitoring system and cameras via network.

[0053] The association information between the switch cabinet and the camera is pre-configured in the intelligent identification device. For example, a mapping relationship is established between the cabinet number "A-01" and the corresponding camera RTSP (Real-Time Streaming Protocol) URL, and saved to the database.

[0054] Load and train the text detection model (ppocrv4_det) and the text recognition model (ppocrv4_rec), as well as the indicator light detection model specifically for indicator light status detection, which is trained based on the YOLOv5 architecture.

[0055] When the staff of the background monitoring system need to perform a closing operation on "Incoming Cabinet No. 1" (cabinet number: A-01), they can send a closing command to the execution device (such as a monitoring and protection device) through interface clicks or command input. At the same time, the background system sends a JSON format request containing the target switch cabinet information to the intelligent identification device, for example: {"cabinet_id": "A-01", "cabinet_name": "Incoming Cabinet No. 1"}.

[0056] After receiving the request from the backend, the intelligent identification device parses out the cabinet number "A-01" of the target switch cabinet. Based on the pre-configured association information, the intelligent identification device finds the RTSP URL of the camera corresponding to cabinet number "A-01", establishes a video stream connection, and begins to acquire the real-time video stream of the camera.

[0057] The intelligent recognition device performs frame analysis on the acquired real-time video stream and calls the ppocrv4_det text detection model to locate the text region in the current frame. For the detected text region, the ppocrv4_rec text recognition model is further called to identify the text content. The identified text content is compared and verified with the cabinet number "A-01" in the backend request. If they match, the camera is determined to be accurately aimed at the target switch cabinet, and the positioning is successful; if they do not match or no valid text is recognized within a timeout, the positioning is determined to have failed, and the backend system is notified to resend the request or manual intervention is required.

[0058] After successful positioning, the intelligent recognition device continuously analyzes the video stream, calls the indicator light detection model to detect the area of ​​the indicator lights on the counter in real time, and identifies their current status (such as color, pattern, etc.).

[0059] In this embodiment, when the red indicator light representing the "closed" state changes from off to on, the intelligent identification device immediately encapsulates this state information into result data, including the indicator light status and a timestamp of the identification time. Simultaneously, the intelligent identification device saves the video footage of the identification moment as a PNG image as visual evidence of the operation result. The intelligent identification device sends the result data and image to the backend monitoring system in real-time or periodically, allowing staff to determine the actual result of the remote control operation.

[0060] When the system receives a termination command from the backend system, or when the recognition process continues for more than a preset time, the intelligent recognition device terminates the current intelligent recognition operation and releases the relevant resources.

[0061] The backend system further processes and analyzes the received status information and images, such as recording operation logs, updating system status, and triggering alarm mechanisms.

[0062] Therefore, through the above specific implementation methods, this invention achieves independent verification and visual confirmation of the remote control results of power system switchgear. This method not only improves the reliability and accuracy of status confirmation, effectively avoiding the risk of misjudgment due to intermediate equipment failure; it also completely replaces the manual on-site verification process, achieving true remote control and unattended operation; simultaneously, it enhances the system's robustness and on-site adaptability; and provides back-end staff with a real-time, transparent means of monitoring the operation process.

[0063] An embodiment of a remote control result confirmation system based on video detection and text recognition like Figure 5 As shown, this embodiment provides a remote control result confirmation system based on video detection and text recognition, including: The back-end system is used to send remote control commands and intelligent recognition requests, and to receive indicator light status information; The camera rotates to a designated preset position according to the PTZ control command of the back-end system to capture the switch cabinet panel; The intelligent recognition device connects to a camera to receive and process the camera's real-time video stream, including text region localization, text content recognition, indicator light status recognition, and sending the recognition results to the backend system. Among them, intelligent identification devices include: The text region localization module calls a text detection model to perform frame analysis on the real-time video stream and locate text regions. The text content recognition module calls the text recognition model to recognize the text content of the located text region; The text content comparison and verification module compares and verifies the identified text content with the unique identifier of the target switchgear in the backend request. After successful localization, the indicator light status recognition module continuously analyzes the video stream and calls the indicator light detection model trained based on the YOLOv5 architecture to detect the status of the indicator lights on the counter in real time. The result sending module encapsulates the identified indicator light status information and the video footage at the time of identification into result data, and sends it to the backend system in real time or periodically.

[0064] Furthermore, the text region localization module adopts a three-stage architecture for text detection model: "PP-LCNetV3 backbone network + DBFPN + PFHead neck + DBHead detection head".

[0065] Furthermore, the text content recognition module adopts a sequence recognition architecture of "SVTR backbone network + Lite-Neck + MultiHead".

[0066] Furthermore, the indicator light status recognition module uses an indicator light detection model based on the YOLOv5 architecture, which is a dedicated signal light recognition model trained using a proprietary dataset of the actual scene.

[0067] Furthermore, it also includes a task termination module, which is used to terminate the current intelligent recognition operation after receiving a recognition termination instruction sent by the background system or after the recognition process has continued for more than a preset time.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A method for confirming remote control results based on video detection and text recognition, characterized in that, include: While sending remote control commands to the execution device for the target switch cabinet, the back-end system also sends a pan-tilt control command to the corresponding camera, causing the camera to rotate to a designated preset position to clearly capture the switch cabinet panel, and sends an intelligent identification request containing the unique identifier information of the target switch cabinet to the intelligent identification device. The intelligent identification device obtains real-time video streams through the RTSP protocol of the corresponding camera by configuring the switch cabinet and corresponding camera according to the needs of the project site. Perform frame analysis on the real-time video stream and call the text detection model to locate the text region in the current frame; The text recognition model is invoked to identify the text content of the located text region; The identified text content is compared and verified with the unique identifier of the target switch cabinet in the background request. If the comparison is consistent, it is determined that the camera is pointing at the target switch cabinet. After successful localization, the video stream is continuously analyzed, and the indicator light detection model trained based on the YOLOv5 architecture is called to detect the status of the indicator lights on the counter in real time. The identified indicator light status information and the video footage at the time of identification are packaged into result data and sent to the backend system in real time or periodically for operation result judgment.

2. The method according to claim 1, characterized in that: The text detection model adopts a three-stage detection model architecture consisting of PP-LCNetV3 backbone network + DBFPN + PFHead neck + DBHead detection head. The PP-LCNetV3 backbone network is used to extract multi-scale features of the image, the DBPPN feature pyramid and PFHead parallel branch fusion structure is used to enhance the feature fusion capability of small-sized, slanted or curved text, and the DBHead detection head is used to output text probability map, threshold map and binary map to achieve accurate segmentation of text regions.

3. The method according to claim 1, characterized in that: The text recognition model adopts a text sequence recognition architecture consisting of an SVTR backbone network, a Lite-Neck, and a MultiHead. The SVTR backbone network segments the text image into character components based on a static visual Transformer framework, extracting local and global features of the text image. The Lite-Neck enhances the interaction of sequence features through SVTR global mixing blocks. The MultiHead includes a CTChead branch for regular text recognition and an NRTRHead branch for irregular text recognition. Finally, the MultiHead outputs the character sequence.

4. The method according to claim 1, characterized in that: The indicator light detection model is a dedicated recognition model for traffic lights trained on a proprietary dataset based on the YOLOv5 architecture and combined with the actual scene. The proprietary dataset includes indicator light samples under different lighting conditions and shooting angles in the actual scene.

5. The method according to claim 1, characterized in that: The intelligent identification device has a pre-stored mapping table that associates the unique identifier of the switch cabinet configured on site with the RTSP address of the camera, that is, the association information between the switch cabinet and the corresponding camera. When a smart identification request containing the unique identifier of the target switch cabinet is received from the backend system, the association mapping table is queried based on the unique identifier of the target switch cabinet to obtain the corresponding camera RTSP address; The intelligent recognition device initiates a real-time video stream connection request to the camera corresponding to the RTSP address via the RTSP protocol. After the connection is established, it continuously receives and parses the real-time video stream data sent by the camera.

6. The method according to claim 2, characterized in that, The construction of the three-stage detection model architecture includes: The PP-LCNetV3 backbone network is constructed by adopting a lightweight convolutional neural network structure. Multiple PP-LCNet modules containing depthwise separable convolution and channel shuffling operations are stacked to extract multi-scale features of the input image layer by layer. A global average pooling layer is introduced at the end of the PP-LCNet module to generate a set of feature maps containing different receptive fields. The DBFPN feature pyramid is constructed by taking the multi-scale feature maps output by the PP-LCNetV3 backbone network as input and constructing the feature pyramid through a bidirectional feature fusion path from top to bottom and bottom to top. Deformable convolutions are inserted in each bidirectional feature fusion path. The output is an enhanced multi-level feature map, with each feature map containing semantic information and spatial details at different scales. The PFHead parallel branch fusion structure is constructed, including the design of a dual-branch parallel structure. Branch 1 is a spatial attention branch, which uses a nonlocal network to capture long-range dependencies and enhance the spatial continuity of text regions. Branch 2 is a channel attention branch, which dynamically adjusts the feature channel weights through the SE module to highlight text-related features. The outputs of the two branches are fused by adding them element by element to generate a fused feature map. Constructing the DBHead detection head involves generating three outputs in parallel based on the fused feature maps: Text probabilistic graph: Generates the probability value of each pixel belonging to a text region through a 1×1 convolutional layer; Thresholding map: An adaptive threshold for each pixel is generated through a 1×1 convolutional layer; Binary image: Combining the text probability map and the threshold map, the final binarized result is generated through a differentiable binarization operation.

7. The method according to claim 3, characterized in that, The construction of the text sequence recognition architecture includes: The SVTR backbone network is constructed by dividing the input text image into non-overlapping character components of a fixed size, with each component serving as the smallest processing unit of the Transformer module; grouping adjacent character components for computation using a local attention module to capture the spatial relationships between components and generate local feature maps; establishing long-distance dependencies among all character components using a global attention module to fuse global semantic information; and fusing the local and global feature maps through concatenation or weighted summation to output a feature sequence containing multi-scale information. Constructing a Lite-Neck involves inserting multiple global blending blocks into the feature sequence; Constructing the MultiHead header includes: The CTCHead branch is designed by employing a connection-based temporal classification (decoder) to map feature sequences to character category probability distributions; and introducing a language model to re-score the CTC output. The NRTRHead branch is designed based on a non-autoregressive Transformer architecture, which generates character sequences through parallel decoding to improve the inference speed of irregular text; a position enhancement module is used to explicitly model character position information. The output weights of the CTCHead and NRTRHead branches are automatically adjusted based on the characteristics of the input text through a dynamic weight allocation layer. The weighted character probability distribution is used to generate the final character sequence through greedy search or bundle search.

8. The method of claim 7, wherein a plurality of global blending blocks are inserted into the feature sequence, each global blending block comprising the following sub-modules: Channel compression module: Reduces the number of feature channels by using 1×1 convolutional layers, thereby reducing computational complexity; Lightweight attention module: It adopts a linear attention mechanism instead of standard self-attention, reducing space complexity; Feature recovery module: Recovers the original number of channels through a 1×1 convolutional layer and introduces residual connections to avoid gradient vanishing; Hierarchical interaction enhancement: A bidirectional long short-term memory network is inserted between adjacent global mixing blocks to strengthen the contextual correlation of sequence features.

9. The method according to claim 4, wherein the construction of the dedicated recognition model for traffic lights includes: For the field scene of indicator lights in power system switchgear, collect raw image data including different lighting conditions, angles, obstructions and indicator light status; An expanded dataset is generated through geometric transformations, color space adjustments, and simulated occlusion to improve the model's generalization ability; rectangular boxes are used to label the indicator light areas, and category labels are defined. The input layer was adjusted by modifying the input size of the original YOLOv5 to match the resolution of the scene image; a coordinate attention module was inserted into the residual block of CSPDarknet53 to enhance the model's sensitivity to the spatial position of the indicator lights; the original 3×3 convolution was replaced with deformable convolution to adaptively adjust the receptive field to adapt to tilted or deformed indicator lights. A small target detection layer is added to the path aggregation network, and shallow features are fused with deep features through upsampling operations; Focal Loss is used instead of standard cross-entropy loss to solve the problem of imbalanced positive and negative sample ratio in indicator light samples; Based on the indicator light size distribution of the on-site dataset, the anchor frame size adapted to small targets is regenerated using the K-means clustering algorithm.

10. A remote control result confirmation system based on video detection and text recognition, characterized in that, include: The back-end system is used to send remote control commands and intelligent recognition requests, and to receive indicator light status information; The camera rotates to a designated preset position according to the PTZ control command of the back-end system to capture the switch cabinet panel; The intelligent recognition device connects to a camera to receive and process the camera's real-time video stream, including text region localization, text content recognition, indicator light status recognition, and sending the recognition results to the backend system. Among them, intelligent identification devices include: The text region localization module calls a text detection model to perform frame analysis on the real-time video stream and locate text regions. The text content recognition module calls the text recognition model to recognize the text content of the located text region; The text content comparison and verification module compares and verifies the identified text content with the unique identifier of the target switchgear in the backend request. After successful localization, the indicator light status recognition module continuously analyzes the video stream and calls the indicator light detection model trained based on the YOLOv5 architecture to detect the status of the indicator lights on the counter in real time. The result sending module encapsulates the identified indicator light status information and the video footage at the time of identification into result data, and sends it to the backend system in real time or periodically.

Images