Image-Based IoT Device Control via Visual Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Controlling multiple Internet-of-Things (IoT) devices in a home or office environment is challenging due to difficulties in identifying the correct device, remembering device names, and navigating through different application interfaces, especially when voice commands are limited in providing general control.
Innovation Solution
A system and method using a mobile device to receive and process images to identify connected devices, providing a unified user interface for control through image recognition, leveraging computer vision and machine learning to map physical devices to virtual control interfaces, allowing users to control devices by scanning images of the devices or linked dummy objects, and associating control functions with device types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple IoT devices are installed in a home environment, then device functionality and control capabilities are improved, but device identification and navigation complexity increase
Solution Approach 1:
The patent introduces an image processing intermediary that acts as a mediator between the user and multiple IoT devices. Instead of directly navigating through multiple device-specific applications, the user captures an image of the target device, and the system processes this image to identify and establish connection with the corresponding device. This intermediary image processing step simplifies the interaction complexity while maintaining the ability to control multiple diverse devices.
Solution Approach 2:
The patent uses a visual copy (image) of the physical device as the control interface entry point. Rather than requiring users to remember device names or navigate through abstract device lists, the system creates a visual copy of the actual device appearance and uses this copy as the key to access device controls. This copying approach maintains device identity recognition while simplifying access.
2Adaptability or versatility
If device-specific applications are used for each connected device, then device control functionality is improved, but user interface consistency and ease of operation deteriorate
Solution Approach 1:
The patent implements a universal image-based control interface that can access and control multiple different types of IoT devices through a single unified application. Instead of requiring separate device-specific applications with different interfaces, the system provides a universal image capture interface that adapts to identify and control various device types (lights, thermostats, cameras, etc.) through consistent interaction methods, thereby improving ease of operation while maintaining broad device compatibility.
3Ease of operation
If voice commands are used for device control, then simple interactions are improved, but general control capabilities and device identification are limited
Solution Approach 1:
The patent substitutes the acoustic/voice-based control mechanism with an optical/image-based control mechanism. While voice commands use acoustic waves for interaction, the patent employs camera-based image capture and processing to identify and control devices. This substitution provides more versatile control capabilities including visual device identification, graphical interface display, and touch-based interaction, while maintaining ease of operation through intuitive visual feedback.
Data Source
AI summary
A system and method in a mobile device for controlling a connected device is disclosed. In one embodiment, the method includes receiving an image; processing the image to identify at least a first image attribute where the first image attribute includes an image feature or an image object; accessing a database of connected devices, each connected device in the database being identified by a linked image and being associated with device metadata describing at least a device type of the connected device; comparing the first image attribute with the linked images corresponding to the connected devices stored in the database; identifying a first connected device having a linked image matching the first image attribute; and in response to the identifying, providing a user interface configured to control the first connected device, the user interface including a control function corresponding to the device type of the first connected device.