visionlink

Abstract

Vision Link is a cross-platform AI-powered computer vision system that enables users to scan physical objects, such as maps, artwork, or product packaging, via a web interface or mobile application. The system utilizes advanced machine learning algorithms to recognize physical objects without the need for visible markers, such as QR codes, and automatically redirects the user to relevant digital content. The mobile app version of Vision Link extends the system's capabilities with GPS-based content delivery, offline scanning, push notifications, and augmented reality (AR) experiences. This technology is designed to provide businesses with real-time analytics on user interactions, offering a seamless bridge between the physical and digital worlds. Vision Link's flexible content management system (CMS) allows businesses to manage, update, and customize their linked content, making it ideal for applications in marketing, retail, real estate, tourism, and more.

Description

Attorney Docket No. REN.001WOPATENT APPLICATION IN THE UNITED STATES PATENT AND TRADEMARK OFFICEVisionLink byDavid Bin YimCheng Lip TanWilson Qiyang PangCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 687,302, filed August 26, 2024, the entire disclosure of which is herein incorporated by reference in its entirety for all purposes.BACKGROUND

[0002] The current methods for connecting physical objects to digital content largely rely on visible markers, such as QR codes or barcodes. While these methods are functional, they often detract from the visual appeal of physical objects and offer limited interaction.Additionally, the majority of QR code systems provide insufficient analytics and engagement data for businesses.

[0003] There is a growing need for a more seamless, markerless solution that allows users to scan physical objects — such as artwork, maps, product packaging, or architectural models — and be redirected to rich digital content without interrupting the aesthetic experience. This solution should be capable of functioning across multiple platforms, including web-basedAttorney Docket No. REN.001WO interfaces, mobile applications and wearable Al eyeglasses, to increase accessibility and flexibility.

[0004] Vision Link is designed to meet these needs by providing an Al-powered computer vision system that allows users to scan physical objects directly via their smartphone's camera or a web-based camera interface. The system then matches the object to a pre-existing database and redirects the user to relevant digital content, such as websites, interactive media, or promotional material. Vision Link also offers real-time analytics and tracking capabilities to provide businesses with detailed insights into user interactions.SUMMARY

[0005] Vision Link is an Al-powered system designed for the seamless scanning of physical objects via both web and mobile applications. It allows users to use all devices from smartphones to the next generation of wearable Al eyeglasses to scan objects such as maps, collectibles, paintings, packaging, or architectural models. The VisionLink system employs a sophisticated multi-stage AI / Machine Learning pipeline for robust object recognition. This pipeline includes:

[0006] • Feature Extraction: Leveraging deep convolutional neural networks (CNNs), such as ResNet or EfficientNet architectures, to extract distinct visual features from captured images. These networks are pre-trained on large-scale image datasets (e.g., ImageNet) and fine-tuned on a proprietary dataset of diverse physical objects. After filtering the top X number of images from vector db, there is a dense local feature extraction using DISK (Dense Image Keypoint) for detailed geometric matching. Followed by geometric verification throught LightGlue neural network matching of DISK keypoints, requiring minimum threshold matches for validation. Once sufficient points match, the output is the image as a match.Attorney Docket No. REN.001WO

[0007] • Object Matching and Recognition: Employing vector databases, L2 distance metric, and similarity search algorithms (e.g., k-nearest neighbors with cosine similarity) to match the extracted features of the scanned object against a database of known objects. This database stores feature vectors, metadata, and linked digital content for each registered physical object.

[0008] • Training and Adaptation: The machine learning models are continuously trained and updated using a combination of supervised and semi-supervised learning techniques. Training data includes various angles, lighting conditions, and instances of each object to enhance recognition accuracy and generalize across different environments. For example, to recognize different editions of a book, the system is trained on images of varying covers, spine designs, and dimensions. For artworks, it learns to identify unique brushstrokes, color palettes, and stylistic elements. The system can be adapted for new object categories through incremental learning, allowing for rapid deployment in new use cases without extensive retraining.

[0009] This disclosure covers Vision Link's capabilities on both web-based and mobile platforms, with particular emphasis on mobile functionalities that take advantage of GPS location services, offline capabilities, push notifications, and augmented reality (AR).

[0010] Key features include:The core innovation of VisionLink lies in its robust markerless object recognition. Unlike traditional methods requiring QR codes or barcodes, VisionLink identifies objects based on their inherent visual characteristics. This is achieved through:

[0011] • Keypoint Detection and Description: Algorithms like SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features), and more recently, learned feature descriptors, are used to identify distinctive keypoints (e.g., comers, edges, unique textures) on the object's surface. These keypoints are then described by numerical vectors thatAttorney Docket No. REN.001WO are invariant to scaling, rotation, and illumination changes.

[0012] • Geometric Verification: After initial keypoint matching, a robust geometric verification step, often involving RANSAC (Random Sample Consensus) or similar pose estimation algorithms, is performed. This filters out incorrect matches and verifies the spatial consistency of the recognized object in 3D space, ensuring accurate identification even from varying viewpoints.

[0013] • Viewpoint Invariance: The training data for the Al models includes images of objects captured from a multitude of angles and distances to ensure high recognition accuracy regardless of the user's scanning perspective. This creates a comprehensive 'fingerprint' for each object in the database, enabling seamless recognition from diverse camera inputs.

[0014] • Integration with mobile devices for scanning using the native camera app.

[0015] • Wearable Al Eyeglasses Integration: Hands-free interaction, real-time recognition, and heads-up display AR overlays.

[0016] • Mobile-specific features such as GPS-based content, AR functionality, and push notifications.

[0017] • A customizable content management system (CMS) for businesses to manage and update linked content.

[0018] • A comprehensive analytics dashboard tracking user engagement, scan frequency, and geographic data.

[0019] • The system can be integrated with existing mobile apps and websites via APIs and SDKs, offering a scalable solution for various industries.BRIEF DESCRIPTION OF DRAWINGS

[0020] Fig. 1 System Architecture Diagram: a flowchart showing the overall architecture of VisionLink. This includes the interaction between the user interface (web and mobile), the AlAttorney Docket No. REN.001WO computer vision engine, the content management system (CMS), and the analytics dashboard.

[0021] Fig. 2 Process Diagram for Web-Based Scanning: a step-by-step diagram showing how a user interfaces with the web-based VisionLink system to scan a physical object.

[0022] Fig. 3 Process Flow Diagram for Mobile App Scanning: A flowchart showing how the mobile application handles object scanning, including any mobile-specific features.DETAILED DESCRIPTION

[0023] Vision Link comprises the following core components:

[0024] 1. User Interface (UI): Accessible through both a web-based interface and mobile application. The web-based UI allows users to scan objects using any internet-connected device’s camera, while the mobile app directly accesses the smartphone's native camera for object scanning.

[0025] 2. Al Computer Vision Engine: An online engine that processes the captured images from both the web and mobile platforms. It employs machine learning algorithms to recognize object characteristics, such as shapes, textures, and patterns, and then matches these against a database of pre-scanned objects.

[0026] 3. Content Management System (CMS): A backend system where businesses can upload and manage the digital content linked to each physical object. The CMS allows easy updates to content and customization of object-digital content pairings.

[0027] 4. Analytics Dashboard: This dashboard provides real-time insights into user interaction. Businesses can access data on scan frequency, user engagement, geographical location of scans, and device type used for the scan. This data is presented in charts and visual analytics that help businesses optimize their marketing strategies.

[0028] 5. Mobile App Integration: The Vision Link mobile app, available for iOS and Android devices, includes native camera access for seamless object scanning. It offersAttorney Docket No. REN.001WO additional features such as:

[0029] • GPS Integration: Visionlink’s mobile application extends its functionality with precise GPS-based content delivery. This feature is particularly valuable for applications in tourism, real estate, and urban exploration.

[0030] 1. Location-Aware Scanning: When the user scans an object, the system simultaneously captures the device’s current GPS coordinates.

[0031] 2. Geofencing and Proximity Triggers: The Content ManagementSystem (CMS) allows content providers to define specific geographical zones (geofences) around physical objects or points of interest. If a scanned object’s location, or the user’s current location, falls within a predefined geofence, the system can trigger the delivery of location-specific digital content. For example, scanning a historical monument in a city could trigger an audio tour relevant to that specific location, or scanning a ‘for sale’ sign could immediately pull up detailed property listings for that exact address, leveraging geotagged content.

[0032] 3. Contextual Content Prioritization: The system can prioritize linked content based on the user’s proximity to associated objects or locations. For instance, if a user scans a generic building, but the GPS indicates they are near a specific historical plaque linked to that building, the content related to the plague will be delivered first.

[0033] • Store Images in Mobile Photo Album: Scanned images can be saved locally on mobile device when there is a loss of internet connection. Once back online, the locally stored images can sync with the app scanning.

[0034] • Predictive Caching: Preloads high-interest content for faster access based on the user’s location, behavior, and preferences.

[0035] • Push Notifications: The app allows businesses to send push notifications based on object scans, promoting special offers, events, or updates.Attorney Docket No. REN.001WO

[0036] • Augmented Reality (AR) Support: Vision Link integrates with AR technology to create immersive digital experiences, such as 3D walkthroughs or virtual annotations on physical objects.

[0037] 6. Wearable Al Eyeglasses Integration

[0038] • Hands-Free Activation: Support for voice commands (e.g., ‘Visionlink, scan this’), gaze detection to select objects in view, or subtle head gestures (e.g., a nod) to initiate scans.

[0039] • Heads-Up Display (HUD) Integration: Seamless overlay of digital content (text, graphics, interactive elements) directly into the user’s field of view, dynamically anchored to the recognized physical object. The UI must be non-intrusive and context-aware.

[0040] • Gaze-Based Interaction: Users can select or interact with overlaid content by focusing their gaze on a specific element for a predefined duration. This enables actions like expanding information or playing a video.

[0041] • Continuous Object Recognition: The system should be able to operate in a continuous mode, passively identifying objects in the user’s environment as they are viewed. This is crucial for applications like tourism or retail navigation.

[0042] • Non-intrusive Notifications: Design for minimal disruption in the user’s field of view. Notifications (e.g., ‘Scanning complete,’ ‘New content available’) should be delivered via subtle visual cues or audio prompts.

[0043] Process Flow

[0044] 1. Web-Based Process:• The user accesses the Vision Link web interface on their device’s browser.• The system prompts the user to grant access to the device’s camera.• The user scans the physical object (e.g., a painting or map) or the image ofAttorney Docket No. REN.001WO the physical object (from a video or photo) using the web-based camera.• The image is sent to the Al Computer Vision Engine, which processes the image and identifies the object by matching it to a pre-scanned database.• The system then redirects the user to a webpage, video, or digital experience associated with that object.

[0045] 2. Mobile App Process:• The user opens the Vision Link mobile app and uses the smartphone's native camera to scan the object.• The app processes the image locally and sends it to the Al engine for recognition.• Depending on the GPS location, the app may deliver location-based content, such as special offers for nearby stores or an augmented reality experience.• If the user is offline, the scan is stored locally, and content is delivered once an internet connection is restored.• The app triggers push notifications based on the user's interactions with specific objects.

[0046] Unique Features and Advantages

[0047] • Markerless Object Recognition: Vision Link does not rely on visible QR codes or barcodes, preserving the aesthetic of physical objects while allowing seamless interaction.

[0048] • Mobile-Specific Features: The mobile app enhances the user experience through GPS-based content, push notifications, offline capabilities, and AR.

[0049] • Comprehensive Analytics: The analytics dashboard provides detailed user data, allowing businesses to optimize their engagement strategies.

[0050] • Cross-Platform Flexibility: The system functions seamlessly across both web and mobile platforms, increasing accessibility for users.

Claims

Attorney Docket No. REN.OOIWOCLAIMSWhat is claimed is:

1. A method for scanning physical objects via a web-based application or a mobile application, comprising: capturing an image of a physical object using a web-based camera or a mobile device’s native camera; processing the image through a cloud-based Al engine to recognize the object; redirecting the user to a corresponding digital content experience based on the recognized object.

2. The method of claim 1, wherein the mobile application further comprises GPS integration for delivering location-specific digital content.

3. The method of claim 1, wherein the mobile application supports offline scanning and preloads content for later delivery once the device is online.

4. The method of claim 1, wherein augmented reality features are provided to overlay digital information on the physical object.

Images