Visual Programming Blocks for Precise Object Behaviors in Enhanced Reality

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Programming behaviors in computer-generated environments is complex and time-consuming, often requiring intricate text-based or node-graph programming that can be difficult for users to navigate.

Innovation Solution

A graphical user interface for visual programming in computer-generated environments, allowing users to interact with programming structures using graphical representations of triggers, actions, and conditional logic, enabling intuitive and efficient behavior definition for objects through drag-and-drop, snapping, and preview/debugging functionalities.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If text-based or node-graph programming is used to define object behaviors, then programming precision and control are improved, but device complexity and ease of operation deteriorate

Engineering Contradiction:
Improveprogramming precisionVSAvoidprogramming structure complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The programming interface is segmented into distinct visual blocks representing different programming constructs (triggers, actions, variables). Each block is a self-contained unit that can be independently manipulated, making the complex programming task manageable through modular visual elements rather than monolithic text code.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The programming interface transitions from one-dimensional text-based or node-graph representations to a two-dimensional block-based visual layout. This dimensional change allows programmers to manipulate code structures through spatial arrangement and visual hierarchy, reducing cognitive load and improving ease of operation while maintaining programming precision.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Manufacturing precision

If text-based or node-graph programming is used to define object behaviors, then programming precision is improved, but ease of operation deteriorates

Engineering Contradiction:
Improveprogramming precisionVSAvoiduser interaction difficulty
Core Design Contradiction:
Manufacturing precisionVSEase of operation

Solution Approach 1:

The traditional mechanical typing and syntax-based interaction system is replaced with a visual drag-and-drop block manipulation system. This substitution eliminates the need for text input, syntax memorization, and error-prone typing, significantly improving ease of operation while maintaining programming precision through visual block assembly.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

Programming blocks can be copied and reused across different contexts within the visual programming interface. This copying mechanism allows common programming patterns to be replicated easily through visual duplication rather than retyping, improving both ease of operation and consistency in programming precision.

Inventive Principle:
Principle #26Copying

3Ease of operation

If visual programming with graphical representations is used, then ease of operation is improved, but programming precision may deteriorate

Engineering Contradiction:
Improveuser interaction simplicityVSAvoidprogramming precision
Core Design Contradiction:
Ease of operationVSManufacturing precision

Solution Approach 1:

The visual programming interface incorporates immediate visual feedback mechanisms that confirm block connections, validate programming logic, and display runtime behavior. This feedback system ensures programming precision by providing real-time verification of block assembly correctness and logical flow, preventing errors before execution.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The system performs preliminary validation and type-checking of block connections during the visual assembly process itself, rather than waiting for runtime execution. This preliminary action ensures programming precision is maintained by catching errors early in the visual programming phase, preventing invalid operations before they occur.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS12360747B1Devices and methods for visual programming
Publication Date: 2025.07.15 APPLE INC
  • US12360747B1 patent drawing
  • US12360747B1 patent drawing
  • US12360747B1 patent drawing

AI summary

Visual programming provides an efficient and intuitive user experience in an enhanced reality computer-generated environment. In some embodiments, a graphical representation of a programming structure is displayed in a computer-generated environment, and various interactions with the graphical representation of the programming structure are provided to define behavior of objects in a computer-generated environment. In some embodiments, the behavior of the object is defined by a trigger and an action. In some embodiments, the various interactions include adding one or more graphical representations of programming objects to the graphical representation of the programming structure, removing one or more graphical representations of programming objects from the graphical representation of the programming structure or from the computer-generated environment, and/or previewing the behavior defined by the graphical representation of the programming structure.