Air writing communication using machine learning
Patent Information
- Application Number
- IN202441034742
- Authority / Receiving Office
- IN · IN
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-05-01
- Publication Date
- 2026-08-04
- Estimated Expiration
- 2044-05-01
AI Technical Summary
Existing human-computer interaction systems that utilize hand gestures are cumbersome and lack efficient visual interpretation, requiring mechanical devices and complex algorithms that are not user-friendly.
An AI-based virtual painter system that uses computer vision with OpenCV and Python to detect a colored marker, allowing users to draw by capturing the motion of the marker with a camera, employing color tracking and morphological operations to create an intuitive interface.
Enables natural and intuitive human-computer interaction by accurately tracing fingertip coordinates and allowing color changes, providing a user-friendly interface for drawing without the need for cumbersome devices.
Abstract
Description
FIELD OF INVENTIONThe present invention generally relates to machine learning system. More specifically, the invention describes the design and development of a Artificial intelligence based virtual painter.BACKGROUND OF THE INVENTIONHuman hand gestures are means of non-verbal interaction among people. They are a type of communication very rich in many ways. The use of hand gesture also provides an attractive alternative to cumbersome interface devices for human-computer interaction (HCl). Integrating the use of hands in HCl would be a great benefit to the user.Visual interpretation of hand gestures can bring the natural means that humans employ to communicate with each other into the HCl. In order to use the hand gesture for the HCl it is necessary to provide the means by which they can be interpreted by the computer.First attempts to solve this problem resulted in mechanical devices that directly measure the hand and arm joint angles and their spatial positions. However, these interaction techniques require cumbersome interface tools, even though accurate results are given.A scientific literature titled "Study on Gesture Recognition System Using Posture Classifier and Jordan Recurrent Neural Network", [Hiroomi Hikawa, et al., 2011] describes a hardware-based gesture classification system combined with Jordan recurrent neural network (JRNN) to make a new type of gesture classifier system. The JRNN is a class of neural network, that can detect dynamic temporal behavior. Each gesture is defined as a sequence of postures and the posture classifier is used toclassify the images composing the gesture. Then the sequence of image indices generated by the posture classifier is fed to the JRNN to classify the dynamic gesture.Another scientific literature titled "Motion-based finger tracking for user interaction with mobile devices", [Jari Hannuksela, et al., 2007] describes A new motion-based tracking algorithm for user interaction with hand-held mobile devices is presented. The idea is to allow mobile phone users to control the device simply by moving a finger in front of a camera. A novel combination of Kalman filtering and the expectation maximization (EM) algorithm is utilized for estimation of two distinct motion components corresponding to the camera motion and the finger motion.One more scientific literature titled "Writing and sketching in the air, recognizing and controlling on the fly", [Sharad Vikram, at al., 2013] describes a way to control and interact with computers by moving fingers in the air. The positions of fingers are precisely captured by a computer vision device. By tracking the moving patterns of fingers, we can then recognize users' intended control commands or input information.Therefore, there exists a need to design and develop an advanced system that should overcome the shortcomings of the currently existing systems.SUMMARY OF THE INVENTIONThe following summary is provided to facilitate a clear understanding of the new features in the disclosed embodiment and it is not intended to be a full, detailed description. A detailed description of all the aspects of the disclosed invention can be understood by reviewing the full specification, the drawing and the claims and the abstract, as a whole.The objective of the present invention is to design and develop an AI based Virtual Painter where we can draw by just capturing the motion of a colored marker with a camera.BRIEF DESCRIPTION OF FIGURESOther features and advantages of the present invention will become apparent from the detailed description of the invention which follows, when considered in light of the accompanying drawings in which:Fig. 1 illustrates the block diagram of the present invention;Fig. 2 illustrates the process flow of the present invention;DETAILED DESCRIPTIONThe principles of operation, design configurations and evaluation values in these non-limiting examples can be varied and are merely cited to illustrate at least one embodiment of the invention, without limiting the scope thereof.The embodiments will be described in detail with corresponding marked references to the drawings, in which the illustrative components of the invention are outlined. The embodiments disclosed herein can be expressed in different forms and should not be considered as limited to the listed embodiments in the disclosed invention. The various embodiments outlined in the subsequent sections are construed such that it provides a complete and a thorough understanding of the disclosed invention, by clearly describing the scope of the invention, for those skilled in the art.It must also be noted that as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Although any systems and methods similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present invention, the preferred, systems and methods are now described.AI Virtual Painter is in where we can draw by just capturing the motion of a colored marker with a camera. One colored object at the tip of the finger is mainly used as the marker. We are here now, using the techniques ofcomputer vision in open cv to build this project. The required language for this project is python due to its more exhaustive libraries and easy to make use of the syntax and but understanding the basics as well as it can be implemented in any open cv supported languages.The color tracking and detection processes are used to achieve the objective of the present invention. The color marker here used is detected and mask is produced. The next steps of morphological operations on the mask produced those are Erosion and Dilation.Erosion makes the impurities present in the mask to get reduced and Dilation further regains the eroded main mask. It creates an interface between human and the system to do the morphological operations. Morphological operations process the digital shapes based on their shapes.Two types of morphological operations are:-> Erosion -> DilationDeveloping an interface between human hand and the system using open cv techniques and python language to pick the color and draw using hand on the developed drawing area.The basic functionalities that are included in this system are:Writing Mode - In this state, the system will trace the fingertip coordinates and stores them.Color Mode - The user can change the color of the text among the various available colors.Experiments conducted:- Frames are read and converted the captured frames to HSV color space (Easy for color detection).- Prepared the canvas frame and applied the respective ink buttons on it.- Adjusted the track bar values for finding the mask of the colored marker.- Pre-processed the mask with morphological operations (Eroding and dilation).- Detected the contours, found the center coordinates of largest contour and kept storing them in the array for successive frames (Arrays for drawing points on canvas).- Finally drawn the points, stored in an array on the frames and canvas.What has been described above includes examples of the disclosed architecture. It is, of course, not possible to describe every conceivable combination of components and / or methodologies, but one of ordinary skill in the art may recognize that many further combinations and permutations are possible. Accordingly, the novel architecture is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims.Without further description, it is believed that one of ordinary skill in the art can, using the preceding description and the illustrative examples, make and utilize the present invention and practice the claimed methods. It should be understood that the foregoing discussion and examples merely present a detailed description of certain preferred embodiments. It will be apparent to those of ordinary skill in the art that various modifications and equivalents can be made without departing from the spirit and scope of the invention.
Claims
1. An Artificial intelligence based Virtual Painter with which the user can draw by just capturing the motion of a colored marker with a camera comprises of; a) One colored object at the tip of the finger is mainly used as the marker; b) The color marker here used is detected and mask is produced; c) Morphological operations such as erosion and dilation process the digital shapes based on their shapes; d) Erosion makes the impurities present in the mask to get reduced and Dilation further regains the eroded main mask; e) Developing an interface between human hand and the system to pick the color and draw using hand on the developed drawing area. f) In writing mode, the system will trace the fingertip coordinates and stores them and in color mode, the user can change the color of the text among the various available colors.
