Electronic Relay Panel Signal Recognition With Mobile AR Guidance

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Interacting with electronic relays in electric power distribution grids is inefficient due to complex user-interface panels with multiple signalling components, requiring experienced personnel and often challenging in poor lighting or remote conditions.

Innovation Solution

A computer-implemented method using a mobile device to capture and process images of the user-interface panel, identifying components and light signals, and providing augmented reality features to assist users in understanding and interacting with the electronic relay.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If multiple signalling components are provided on the user-interface panel, then the electronic relay can provide comprehensive operating information, but the complexity of the user-interface panel increases and becomes difficult to interpret

Engineering Contradiction:
Improveoperating information completenessVSAvoiduser-interface panel complexity
Core Design Contradiction:
Loss of informationVSDevice complexity

Solution Approach 1:

A mobile device application serves as an intermediary between the electronic relay's complex user-interface panel and the user. The application captures images of the panel, automatically identifies and interprets multiple signalling components and light signals, then presents simplified information to the user, eliminating the need for them to directly interpret the complex panel displays.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces manual visual inspection and interpretation of physical signalling components with automated image processing and signal analysis. The mobile device's camera and processing algorithms substitute for human eyes and brain in interpreting the panel's light signals and component states.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If technical manuals are consulted to interpret light signals and identify components, then accurate information can be obtained, but the interaction time and efficiency are significantly reduced

Engineering Contradiction:
Improvesignal interpretation accuracyVSAvoidinteraction time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The mobile device application performs self-service by automatically capturing images of the user-interface panel, identifying components and light signals, and interpreting their meanings without requiring external reference to technical manuals. The system serves itself by integrating the interpretation functionality directly into the mobile application.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The application pre-loads and stores interpretation rules and component information in its database, allowing it to immediately interpret signals without requiring real-time consultation of external manuals. This preliminary preparation enables instant interpretation when images are captured.

Inventive Principle:
Principle #10Preliminary action

3Adaptability or versatility

If electronic relays are installed in remote or hard-to-reach positions, then grid coverage is improved, but user accessibility and interaction become difficult

Engineering Contradiction:
Improveinstallation flexibilityVSAvoiduser accessibility
Core Design Contradiction:
Adaptability or versatilityVSEase of operation

Solution Approach 1:

The mobile device application provides universal functionality that works with electronic relays regardless of their installation location. By using the user's existing mobile device and leveraging camera-based image capture, the system maintains ease of operation whether the relay is in a readily accessible location or in a remote, hard-to-reach position.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The mobile application acts as a mediator that enables interaction with the electronic relay without requiring the user to physically approach or closely examine the relay's user-interface panel. Users can capture images from a distance and receive interpreted information on their mobile device, making relays in remote positions equally accessible.

Inventive Principle:
Principle #24Intermediary (Mediator)

4Ease of operation

If augmented reality features are used to guide users, then ease of operation is improved, but processing requirements and device complexity increase

Engineering Contradiction:
Improveuser guidance capabilityVSAvoidsoftware processing complexity
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

The application creates a digital copy or representation of the physical user-interface panel by capturing its image and overlaying augmented reality elements. This virtual copy guides users through procedures by showing annotations and instructions directly on the captured image of the actual panel, maintaining simplicity while providing guidance.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS11961292B2Computer-implemented method for assisting a user in interacting with an electronic relay for electric power distribution grids
Publication Date: 2024.04.16 ABB SPA
  • US11961292B2 patent drawing
  • US11961292B2 patent drawing
  • US11961292B2 patent drawing

AI summary

Disclosed herein is a computer-implemented method to assist a user in interacting with an electronic relay for electric power distribution grids. The method provides for identifying light signals emitted by one or more user-interface components of the user-interface panel of the electronic relay by processing images of said user-interface panel, which have been captured by the camera of a mobile computerized device. The method further provides for showing information, which describes the identified light signals emitted by said user-interface components on the display of said mobile computerized device.