Display control system, method, display device, and controller

Abstract

The present disclosure relates to the technical field of controlling an image by linking a controller and a display device. The controller wirelessly transmits orientation and operation data, and relative position data with respect to the display device. The display device specifies an instruction position on the basis of said data, generates an overlay image related to the instruction position, and displays the overlay image on a display screen.

Description

Display control system, method, display device and controller

[0001] The present disclosure relates to the technical field of display control, and more particularly to a system and method for controlling images and information on a display using a user-operated controller.

[0002] In conventional display control systems, a sensor bar or marker is typically installed on the display side to achieve an accurate interface between the controller operated by the user and the display.

[0003] For example, the Nintendo Wii (registered trademark) remote control has a sensor bar attached to the top of the television that detects infrared light emitted from the remote control to enable pointing operations. VR devices also employ a method in which a camera sensor is attached to the display and infrared LEDs or other markers are attached to the controller.

[0004] These methods require additional hardware for the controller and display, which tends to complicate the system configuration and make installation more difficult.

[0005] This disclosure relates to a technology that aims to improve the operation accuracy and usability of conventional technology, and in particular, focuses on providing accurate pointing operations on a display based on real-time communication between a controller and a display device.

[0006] The present disclosure includes a technology in which: (1) a controller wirelessly transmits posture and motion data, as well as relative position data to a display device; (2) the display device identifies a pointing position based on the data received from the controller; and (3) an overlay image generated based on the pointing position is displayed on the display, thereby providing an intuitive and accurate operation experience throughout the system.

[0007] This disclosure enables users to operate the display with a simple configuration without adding special hardware to the display. Furthermore, the sensors integrated into the controller enable highly accurate acquisition of posture and motion data, as well as relative position data, enabling accurate identification of the indicated position on the display. Furthermore, the controller configuration is simple, and by utilizing, for example, a combination of a 9-axis sensor and a laser distance sensor, highly accurate data acquisition and communication are achieved without the need for special additional hardware.

[0008]

[0013] Further aspects and advantages of the present disclosure will become readily apparent to those skilled in the art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the present disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

[0009] 1 is a block diagram showing the configuration of a controller according to an embodiment; FIG. 2 is a diagram showing a specific shape of a controller according to an embodiment, where (A) is a front view and (B) is a rear view; FIG. 3 is a block diagram showing the configuration of a display device according to an embodiment; FIG. 4 is a flowchart showing a process flow performed between a controller and a display device according to an embodiment, illustrating a series of processes related to displaying a base image, position resetting, and real-time image manipulation; FIG. 5 is a schematic diagram showing a position reset process according to an embodiment, illustrating a scene in which the controller is directed toward a reset cursor displayed on the screen of the display device; FIG. 6 is a schematic diagram showing a real-time manipulation process according to an embodiment, illustrating a scene in which an overlay image displayed on the screen of the display device is displayed according to a position pointed by the controller; FIG. 7 is a schematic diagram showing a position reset process on multiple display devices according to an embodiment; FIG. 8 is an example of a state in which a reset cursor is displayed on a specific display device and the controller is being operated toward the cursor; FIG. 9 is a schematic diagram showing a real-time manipulation process on multiple display devices according to an embodiment; FIG. 10 is an example of a state in which the controller specifies a pointed position and an overlay image is displayed on the display device; FIG. 11 is a flowchart showing a process flow performed between a controller and a display device according to an embodiment, illustrating a series of processes related to displaying a base image, position resetting, and real-time image manipulation. 1A-1C are schematic diagrams illustrating a position reset process in accordance with one embodiment, where the controller is pointed at a display device and a reset cursor is displayed on a base image; 1B-1C are schematic diagrams illustrating real-time operation in accordance with one embodiment, where the controller is close to the display device and an overlay image is generated and displayed for close-range operation; and 1C-1C are schematic diagrams illustrating real-time operation in accordance with one embodiment, where the controller is far from the display device and an overlay image is displayed that adjusts depending on the distance;FIG. 10 is a schematic diagram showing a scene in which the magnification of an overlay image is changed using the operation ring of a controller according to one embodiment, illustrating an example of adjustment of the overlay image by a user operation; FIG. 11 is a flowchart of a calculation process performed between a pointing device and a display device according to one embodiment; FIG. 12 is a description of an embodiment; FIG. 13 is a description of an embodiment; FIG. 14 is a description of an embodiment; FIG. 15 is a description of an embodiment; FIG. 16 is a description of an embodiment; FIG. 17 is a description of an embodiment; FIG. 18 is a description of an embodiment; FIG. 19 is a description of an embodiment; FIG. 19 is a description of an embodiment; FIG. 19 is a description of an embodiment; Detailed Description of the Invention

[0010] As used herein, the term "controller" refers to a device operated by a user and capable of transmitting control signals to a display device via wireless communication. The controller may include a sensor module that generates attitude and motion data and relative position data, an input interface that accepts user input, and a communication unit. In this specification, the terms "controller" and "pointing device" are used interchangeably.

[0011] The term "display device" as used herein refers to a device equipped with a display screen for presenting image information to a user and capable of controlling the display content based on data received from a controller via wireless communication. The display device may include a communication unit, a processor, and a display control unit.

[0012] As used herein, the term "base image" refers to an image that is displayed as the initial or background on a display device. This image can be static (still image) or dynamic (moving image). Furthermore, the base image is used as a basis for the generation of overlay images and serves to provide a visual reference for the user.

[0013] In some embodiments, the base image may include a natural landscape, a cityscape, outer space, or an abstract pattern, allowing the display device to function more than just an information display device, but also as an artificial window. For example, by displaying a landscape image on the display device, a wall-mounted display provides the visual effect of the landscape being physically present outside.

[0014] The foundation image may be pre-stored in storage (e.g., a non-transitory storage medium) within the display device, or may be acquired from an external source, including downloading via the internet, streaming, or a cloud-based service. Dynamic content that updates in real time (e.g., live video, interactive information, etc.) may also be used as the foundation image.

[0015] It is also possible to link multiple display devices to operate in a unified manner. In this configuration, the multiple display devices understand their relative positions and display consecutive images. In one embodiment, a method is employed in which layout information is manually input to each display device. In another embodiment, the relative positions can be automatically understood through communication between the display devices.

[0016] By linking multiple display devices, a vast landscape image can be displayed as if it were a single continuous screen. This continuity allows users to have a visual experience similar to looking out a window. Furthermore, the base image is not limited to landscape images; a wide range of content can be used, including educational content (e.g., maps, scientific videos, etc.), entertainment (e.g., movies, games, etc.), and interactive information (e.g., news, weather forecasts, etc.).

[0017] The generation and selection of the foundation image may be performed by the display device's control processor or by a cloud-based service, allowing users to select or customize foundation images based on a particular theme or need.

[0018] By integrating multiple display devices, users can enjoy an immersive and interactive experience, enhancing the user experience in a wide range of applications, including entertainment, education, presentations, and advertising.

[0019] As used herein, the term "overlay image" refers to an image that is an enlarged version of a part or all of a base image and is displayed superimposed on the base image on a display device.

[0020] Overlay images come in a variety of forms and uses, examples of which include, but are not limited to:

[0021] 1. Cursor The cursor as an overlay image is a display that indicates a specific position on the base image, and the user can move that position by operating the controller.

[0022] 2. Enlarged display of part of the base image An overlay image, which enlarges and displays a part of the base image, is used to present details of a location specified by the user. For example, by enlarging a part of a map image, more detailed information can be provided.

[0023] 3. Supplementary information associated with the base image: Information related to a specific element on the base image can be displayed as an overlay image. For example, when a bird in the base image is pointed to, the bird's species name, ecological information, or related images and text may be displayed.

[0024] 4. Menus or Operation Guides Menus or operation guides may be displayed as overlay images to assist the user in their operation, including options for changing display device settings or selecting specific functions.

[0025] 5. Real-time Information The overlay image may display real-time information generated in response to the content of the base image or user operations. For example, if the base image is a video, data related to the time axis of the video is displayed as an overlay.

[0026] 6. Presenting Multiple Layers of Information Overlay images can also be used to simultaneously display different types of information as multiple layers on top of a base image, for example, a map image may be overlaid with temperature data, traffic information, or detailed information related to a user-selected point.

[0027] 7. Interactive Elements When a user selects or points to a specific element on the base image, a responsive overlay image may be generated based on that selection. For example, when a product is selected on an image of a product catalog, the price or specifications of that product may be displayed.

[0028] 8. Highlighting Overlay images may be used to highlight specific areas or elements on the base image, for example, to highlight specific keywords in a sentence or to indicate specific locations on a map using color.

[0029] 9. 3D or Depth Information If the base image can be displayed in 3D, depth information or a 3D model may be added as an overlay image.

[0030] 10. Annotations Overlay images may be used to display annotations or notes that a user adds on top of a base image, where the text or graphics entered by the user are displayed in association with the base image.

[0031] In addition to these examples, other forms of overlay images may be used depending on the particular application and embodiment.

[0032] The term "pointing position" as used herein refers to a specific position on the display screen that the user points to by manipulating the controller. This position is identified within the display coordinate system based on the attitude and movement data and the relative position data.

[0033] As used herein, the term "display coordinate system" refers to a reference coordinate system for defining positions and / or orientations within a display device, where the origin and the direction of each axis are set based on the geometric structure of the display screen.

[0034] As used herein, the term "display control unit" refers to a component that has the function of displaying an image on a display device or overlaying an image on a substrate image based on instructions from a processor.

[0035] As used herein, the term "attitude and motion data" collectively refers to information about the attitude (e.g., tilt, rotation, etc.) and / or motion (e.g., acceleration, movement, etc.) of the controller. In some embodiments, this data is obtained using one or more of the following example modules:

[0036] In some embodiments, a 9-axis sensor may be used. The 9-axis sensor is a measurement module that combines a 3-axis accelerometer, a 3-axis gyroscope, and a 3-axis geomagnetic sensor, and is capable of detecting the acceleration, rotational speed, and geomagnetic information of the controller. This makes it possible to comprehensively acquire attitude and movement data of the controller. The attitude and movement data may include the controller's Euler angles (pitch (θ): rotation angle around the X-axis, roll (φ): rotation angle around the y-axis, yaw (ψ): rotation angle around the z-axis), a quaternion (q = (q0, q1, q2, q3)), and a rotation matrix (R = r_ij).

[0037] Furthermore, in other embodiments, the following modules may be used: Acceleration sensor: Detects acceleration in the X, Y, and Z axes and measures movement and vibration. Gyroscope: Detects angular velocity and measures rotational movement and attitude changes. Geomagnetic sensor: Detects the orientation of the controller and measures direction and orientation. IMU (Inertial Measurement Unit): A unit that combines an acceleration sensor, a gyroscope, and in some cases a geomagnetic sensor, and is suitable for comprehensively acquiring attitude and movement data. Optical motion capture sensor: Tracks the movement and attitude of the controller using infrared or visible light. Ultrasonic sensor: Measures the distance and movement direction of the controller using reflected ultrasound. LiDAR: Uses laser light to measure the position and movement of the controller with high precision. Sonic tracking system: Measures the position and movement of the controller using the time lag between the propagation and reception of sound waves. Pressure sensor: Detects pressure or vibrations received by the controller and complements movement data.

[0038] The term "sensor module" used in this specification refers to a configuration including one or more of the above-mentioned sensors, and includes not only the use of a single module but also a configuration combining multiple sensors. For example, an acceleration sensor and a gyroscope may be combined to measure both movement and rotation. For example, an optical sensor and a geomagnetic sensor may be used together to measure position and orientation with high accuracy.

[0039] These modules may be located within the controller or may be used in combination with externally installed sensors, and are used to acquire posture and movement data in response to user operations and control the operation of the display device and related systems.

[0040] As used herein, the term "relative position data" refers to data indicating the distance and / or positional relationship between the controller and the display device. In some embodiments, this data is measured using laser distance sensors, ultrasonic sensors, infrared sensors, or other suitable technologies. These technologies are used to accurately detect the distance to a reflector (obstacle) in the direction the controller is pointing. For example, laser distance sensors are suitable for high-precision measurements over long distances, while ultrasonic sensors and infrared sensors are suitable for medium- and short-range measurements.

[0041] As used herein, the term "laser distance sensor" refers to a module that uses laser light to measure the distance between a controller and a display device or an object. This sensor includes a laser emitter and a photodetector and may employ a time-of-flight (ToF) method, a phase difference method, or other suitable measurement method. Furthermore, the laser distance sensor is configured to detect reflectors (obstacles) in the direction the controller is pointing, and may be used to obtain position information for the display device or to specify an operation target.

[0042] Because the laser distance sensor measures the distance from the controller to a reflector, if the target is not the screen of the display device, the distance from the display device may not be obtained. Even in this case, the position reset process prompts the user to point the controller at the reset cursor, so this does not pose a major problem during normal use. By accurately pointing the controller based on the position reset cursor, the initial position of the controller and the indicated position on the display device are synchronized, and accurate position information is obtained. Furthermore, if the user feels unnatural behavior during operation, the position reset can be easily performed again, so the operating experience is maintained.

[0043] Similarly, if the user moves significantly relative to the display device or the operating range of the controller is changed, and the pointed position is not set appropriately, the user can re-execute the position reset to resolve the problem. In this case, the display device has a function to perform a default correction operation to assist the user's operation.

[0044] The term "relative position data" used in this specification is obtained by using these sensor technologies alone or in combination. This allows the positional relationship between the controller and the display device to be accurately determined, providing an environment in which the user can operate intuitively. This design is an important element supporting high-precision operation in position reset and real-time operation.

[0045] As used herein, the term "control ring" refers to a component located on the outer periphery of a controller that detects rotational operation by a user and is used to control the operation of a display device or related system based on that operation.

[0046] In some embodiments, the control ring may be used to achieve a variety of functions, such as: Controlling the zoom function: Enlarging or reducing the content on a screen or in a window on the display device. Adjusting the volume: Increasing or decreasing the volume of the display device or an external system depending on the amount of rotation of the control ring. Navigation within a screen or virtual space: Rotating the control ring allows movement in a specific direction within the screen or virtual space. Selecting a menu or switching operation modes: Rotating the control ring is used to select a specific item from multiple options or to switch operation modes.

[0047] The operation ring has a function of detecting the direction of rotation (e.g., clockwise or counterclockwise) and the amount of rotation, and its operation is controlled based on the user's intuitive operation. Specifically, the operation ring has convex and concave portions arranged at equal intervals, and a configuration may be adopted in which changes in these portions are detected using an optical, magnetic, or mechanical encoder or sensor. For example, when an optical encoder is used, the rotational operation can be obtained as a signal of 0 or 1 by detecting the difference in light reflection by the convex and concave portions.

[0048] As used herein, the term "control ring" is not limited to any particular application or sensing technology and may be used to control zoom, volume control, navigation, menu selection, or other suitable functions, including optical, magnetic, or mechanical encoders, as well as other suitable technologies.

[0049] The term "position reset button" as used in this specification refers to a part of the input interface provided on the controller that is used, when pressed, to reset the initial position of the controller and set the initial attitude and movement data and initial relative position data.

[0050] As used herein, the term "cylindrical controller" refers to a controller having an elongated shape and configured to be used with its longitudinal direction facing a display device. This controller may include a hollow structure or a configuration with open ends.

[0051] In some embodiments, the controller is equipped with a nine-axis sensor and a laser distance sensor, providing users with an intuitive and versatile experience for a variety of applications. The possible uses of this controller are described below.

[0052] First, this controller is suitable for applications in the gaming and entertainment fields. For example, by using it as a virtual tool like a lightsaber (product name, trademark, Disney), a fishing rod, or a gun, it provides a realistic feel in action games and simulation games. The controller's nine-axis sensor accurately detects the user's movements and reflects them in the movement of an on-screen pointer or character. Furthermore, a laser distance sensor can be used to reflect the user's distance information in in-game interactions.

[0053] Second, this controller can also be used for measurement purposes. For example, when measuring the length of a room or the dimensions of furniture, the distance can be measured with high precision by pointing the controller at a wall or object. Also, by holding the controller vertically and pointing it from the floor to the ceiling, the height of a room can be measured. Similarly, by pointing the controller from the user's feet toward their head, the height can be measured. These measurements are based on high-precision distance measurement technology using a laser distance sensor.

[0054] Third, the controller is useful for educational, presentation, and professional applications. For example, when displaying maps or scientific data, the controller can be used to zoom in on a specific area. Furthermore, supporting virtual annotation or selection on the display device allows for efficient sharing or presentation of information.

[0055] This controller combines a 9-axis sensor and a laser distance sensor to achieve both precision and flexibility of operation. Even without a gyro sensor, the 9-axis acceleration sensor and geomagnetic sensor can adequately detect the user's posture and movement. Furthermore, the laser distance sensor contributes to complementing position information and distance measurement, enabling a wide range of application scenarios.

[0056] Furthermore, by utilizing interfaces such as the control ring and buttons, it is possible to compensate for the lack of precision in game control and measurement applications. For example, adjusting the magnification rate with the control ring enhances visual feedback and improves the user's operating experience. In this way, this controller is an interface device that combines multifunctionality and flexibility, making it suitable for a wide range of applications.

[0057] 1 is a block diagram showing the configuration of a controller 100 according to the present invention. The controller 100 is equipped with multiple user interfaces, sensors, and control modules for communicating with and operating a display device. These components are designed to achieve intuitive and efficient operation.

[0058] The controller 100 includes multiple components. The user interface 120 has multiple buttons or operation rings for user operation and generates operation signals. This includes a power button 121 for turning the controller's power on or off, a cross button 122 that allows directional operation up, down, left, and right, a center button 123 for performing selection and confirmation operations, an option button 124 for displaying a menu and calling up a setting screen, and a menu button 125 that provides specific screen switching and shortcut functions. The controller 100 also includes a trigger button 126 for resetting and performing auxiliary operations, and an operation ring 127 for zooming, adjusting the volume, and selecting menu items.

[0059] The network interface 130 is a module that transmits and receives data to and from the display device via wireless communication, and establishes communication between the controller 100 and the display device via an antenna 131 .

[0060] The sensor module is equipped with a nine-axis sensor 132 that measures the controller's attitude and movement data, and a laser distance sensor 133 that measures the distance to a reflector in the direction the controller is pointing. These sensors provide a highly accurate interface based on user operation.

[0061] The storage 113 is a non-volatile storage medium that stores software or programs for controlling the operation of the controller 100. On the other hand, the memory 112 is a volatile storage medium that temporarily stores data in real time, and the central processing unit 111 comprehensively manages the operation of each component within the controller, thereby generating a signal to be sent to a display device based on a user input.

[0062] These components are interconnected via a system bus 140 and are designed to enable efficient data flow and synchronous operation, enabling the controller 100 to achieve high-precision and rapid response to user operations.

[0063] 2A and 2B are diagrams showing a specific shape of the controller 100 shown in FIG. 1, with (A) being a front view and (B) being a rear view.

[0064] The controller 100 has a cylindrical shape, with its longitudinal direction facing the display device. This cylindrical structure is hollow, allowing the user to view the display device or an object through the hollow. This design provides the user with an intuitive and immersive operating experience.

[0065] A number of buttons are arranged longitudinally on the front of the controller 100, including a power button 121, a cross button 122, a center button 123, an option button 124, and a menu button 125. A trigger button 126 is provided on the back, allowing for reset and auxiliary operations.

[0066] An operation ring 127 is provided at one end, and this ring can be rotated to adjust zoom or volume or select menu items. The ring is positioned within the user's natural reach, providing comfortable operation. Tactile feedback allows the user to perform operations without the need for visual confirmation.

[0067] The hollow cylindrical structure also provides users with a new operating experience. By looking into the display device through the controller, users can feel as if they are using a telescope. This interaction has the effect of increasing the enjoyment and practicality of interactive games, learning, and other applications.

[0068] Furthermore, the user interface components are arranged longitudinally along the cylindrical structure, which fits the shape of the hand and makes it easy to operate with one hand. The uniform spacing between buttons prevents accidental operation and enhances the unity of the design, giving the overall product a refined appearance.

[0069] The cylindrical shape of the controller 100 shown in Fig. 2, the layout of the user interface, and the position of the operation ring 127 are optimized to allow a user to intuitively and efficiently operate the display device. This configuration improves the user experience, especially in applications where interactive operation is required.

[0070] 3 is a block diagram showing the configuration of a display device 300 according to an embodiment. The display device includes a plurality of components for displaying images or information in response to user operations.

[0071] The display device 300 includes a display panel 320 for visually outputting images or information. The display panel 320 may be based on a liquid crystal display (LCD), organic light emitting diode (OLED), or other suitable display technology and is designed to display visual information at high resolution. Furthermore, the display panel 320 is capable of displaying still images, moving images, and an interactive graphical user interface (GUI).

[0072] The display device 300 is provided with a network interface module 330 for transmitting and receiving data to and from external devices via wireless communication, which communicates with a controller or other external devices via an antenna 331.

[0073] Furthermore, the display device 300 includes a central processing unit 311 for executing data processing, a memory 312 for real-time data processing, and a non-volatile storage 313 for storing the operating logic or firmware of the entire system. This storage 313 is configured as a non-transitory storage medium and stores software and programs for controlling the operation of the display device. This software may include image processing algorithms, user interface operation logic, network communication protocols, etc. This storage also allows for system firmware updates and data log storage.

[0074] The display device 300 includes a graphics processing unit 314 for optimizing image processing and achieving high-speed rendering. The GPU 314 controls the display of images on the display panel 320 and plays a role in improving the responsiveness of the user interface.

[0075] These components are interconnected via a system bus 340 and are designed to enable efficient data flow and synchronous operation, which enables the display device 300 to achieve high-precision and rapid processing of signals from the controller accompanying user operations.

[0076] With the above configuration, the display device 300 according to one embodiment provides an interactive operation experience in cooperation with a controller and realizes an intuitive and user-friendly interface. This device is applicable to a wide range of uses, including home entertainment, education, presentations, and more.

[0077] 4 is a flowchart showing a process flow between a controller and a display device according to an embodiment, which is mainly divided into a position reset process (S410) and a real-time image manipulation process (S420).

[0078] In the position reset process (S410), when a position reset operation by the user is detected (S411), the controller transmits the information to the display device. The display device displays a reset cursor on the screen (S412), and the user points the controller at the cursor.

[0079] At this time, the display device may output (request) to the user a request to point the controller at the reset cursor in the form of an image, sound, vibration of the controller, etc. Alternatively, this request may be written in the instruction manual that the user reads when purchasing the display device.

[0080] The controller acquires initial data using the built-in 9-axis sensor and laser distance sensor (S413), and the initial position of the controller is calculated based on this data (S414). Here, the laser distance sensor measures the distance D0 in the direction the controller is pointing, but this measurement result is not used to determine the accuracy of the orientation toward the cursor. The reset cursor is provided to the user as a reference point, and it is estimated that the controller is facing the cursor due to user operation.

[0081] In the real-time process (S420), the controller acquires current sensor data (S421), and the display device calculates pointed coordinates based on this data (S422). Next, the display device generates an overlay image based on the pointed coordinates (S423) and displays it superimposed on the position (S424). This allows the user to intuitively operate the display device in real time.

[0082] 5A and 5B are schematic diagrams showing position reset and real-time operation. In FIG. 5A, a reset cursor 522 is displayed on the screen 511 of a display device 510, and a controller 550 is pointed at the reset cursor. The controller 550 measures the pitch angle θ0, yaw angle ψ0, and roll angle φ0 as part of the attitude and movement data acquired by the nine-axis sensor. This data provides detailed movement information of the controller and is used to identify the indicated coordinates.

[0083] 5B shows a scene of real-time operation, in which an overlay image 523 is displayed on the screen 511 of the display device 510. The overlay image 523 may be, for example, a portion of a zoomed map, a cursor, or text and / or icons related to an object. This image is intuitively displayed on the screen in response to a user's operation, improving operability.

[0084] Data is transmitted and received between the controller 550 and the display device 510 via wireless communication. High-precision, low-latency communication allows sensor data from the controller to be transmitted to the display device 510, which then performs necessary calculations based on this data. This configuration allows the user to experience smooth, real-time operation.

[0085] 6A and 6B are schematic diagrams illustrating position reset and real-time operation when multiple display devices operate in cooperation with each other. In this embodiment, a display device set 610 is composed of three display devices 610a, 610b, and 610c. To determine the relative positions of the display devices in this set, an installer can manually input position information after installation, or the display devices can communicate with each other and automatically determine their relative positions. This allows the display device set 610 to operate the multiple display devices as if they were a single screen.

[0086] 6A shows a scene from the position reset process, illustrating a state in which the controller 650 sets an initial position. A reset cursor 622 is displayed on the screen 611b of the display device 610b in the display device set 610, and the user operates the controller 650 while pointing at this reset cursor. The controller 650 acquires attitude and movement data using a nine-axis sensor, and the initial position is calculated based on this data. The nine-axis sensor here is an integrated combination of an acceleration sensor, a gyroscope, and a geomagnetic sensor. The pitch angle θ0, yaw angle ψ0, and roll angle φ0 are illustrated as examples, but these are only a portion of the acquired data.

[0087] 6B illustrates a real-time operation process, exemplifying a state after the controller 650 has been moved and rotated. An overlay image 623 is displayed on the screen 611c of the display device 610c of the display device set 610. In this scene, the controller 650 acquires current attitude and movement data (e.g., pitch angle θ1, yaw angle ψ1, roll angle φ1), and the display device set 610 calculates pointing coordinates based on the data. During this process, real-time data transmission and reception is performed between the controller 650 and the display device 610, enabling quick and intuitive operation.

[0088] 7 is a flowchart showing the process flow between the controller and the display device according to one embodiment, which consists of three main parts: displaying the base image (S701), a position reset process (S710), and a real-time image manipulation process (S720).

[0089] First, a display device displays a base image on a screen (S701). The base image is an image used as a background, and may include a still image, a moving image, or a user interface element. This base image is displayed in combination with an overlay image generated in a subsequent process, and serves as a base for a user to perform interactive operations.

[0090] In the position reset process (S710), the display device displays a reset cursor at a predetermined position on the screen (S712). The user operates the controller toward this reset cursor, and initial data is acquired using sensors within the controller (e.g., a 9-axis sensor and a laser distance sensor) (S713). Based on this data, the initial position of the controller is calculated (S714). The reset cursor functions as a reference point for the user, and by accurately pointing the controller, the initial position of the controller is synchronized with the reference point on the display screen. This process enables highly accurate calculation of indicated coordinates in subsequent operations.

[0091] In the real-time process (S720), the controller acquires current sensor data (S721), and the display device calculates pointed coordinates based on the data (S722). Next, an overlay image is generated (S723) according to the distance between the pointed coordinates and the controller and the magnification ratio set by the operation ring, and is displayed superimposed on the base image (S724). The overlay image is designed to grow in size as the distance increases, and further fine adjustments can be made using the operation ring. This allows for dynamic interaction based on distance or operation.

[0092] The combination of the base image and the overlay image allows the user to have an intuitive and interactive operation experience. In this embodiment, when the user holding the controller actually moves their body or hands, the position of the pointing device on the screen changes dynamically, and the size of the overlay image is adjusted accordingly in real time. This system configuration allows the user to intuitively feel the correspondence between their physical actions and the visual results, providing a seamless feeling of operation.

[0093] For example, as a user moves closer to the display device while holding the controller, the overlay image becomes smaller, and more detailed information is enlarged on the screen. Conversely, as the user moves back, the overlay image becomes larger, and the overall image expands on the screen. This action creates the feeling that the user is "zooming in" or "zooming out" on the image or information themselves. This experience, by linking physical movement with changes on the screen, provides a sense of immersion to the user and provides a more natural feel to the control than traditional button-operated controls. In some embodiments, the size of the overlay image may be determined depending on the distance between the controller and the display. For example, the larger the distance, the smaller the size of the overlay image, or vice versa. In some embodiments, the magnification of the overlay image may be determined depending on the distance between the controller and the display. For example, the smaller the distance, the larger the magnification of the overlay image, or vice versa. In some embodiments, the size and magnification of the overlay image may be determined depending on the distance between the controller and the display.

[0094] Furthermore, as the user moves their hand to point the controller at a different part of the display screen, the pointing position also moves in real time, instantly switching the range of target information or image. This allows the user to freely explore the content on the display through their own movements. This type of operation is particularly convenient when dealing with a wide range of information or large images, and provides the user with the satisfaction of being able to directly control the information through their own movements.

[0095] Another feature of this system is that users can operate it using their own body movements, without using a control ring or buttons. This design not only provides intuitive operability, but also a new interaction experience that incorporates a sense of kinesthetic sensation. For example, in educational settings, users can move closer to the display to check the details of an image, or during presentations, they can view the entire screen from a distance, creating a more natural operating environment.

[0096] 8A-8D are schematic diagrams illustrating the interaction between a controller and a display device in one embodiment, illustrating a step-by-step process from position reset to real-time manipulation and even adjusting the magnification of an overlay image.

[0097] 8A shows the position reset process. A base image 821 is displayed on a screen 811 of a display device 810, and a reset cursor 822 is displayed above it. In this process, the user operates the controller 850 toward the reset cursor, and attitude and movement data is acquired using a sensor (e.g., a 9-axis sensor) within the controller. This data includes a pitch angle θ0, a yaw angle ψ0, and a roll angle φ0, and the initial position of the controller is calculated. The reset cursor serves as a reference point for the user's operation, and the initial position is set by the user accurately pointing the controller.

[0098] 8B shows close-range operation in a real-time operation process. When the controller 850 is close to the display device (distance D1), the size of the overlay image 823a displayed on the display screen 811 is adjusted to a smaller size. In this situation, the controller acquires current attitude and movement data (e.g., pitch angle θ1, yaw angle ψ1, roll angle φ1), and the display device 810 calculates the indicated coordinates based on this data. This real-time data transmission and reception allows the user to perform accurate operation.

[0099] 8C shows an operation performed when the controller 850 is positioned far from the display device (distance D2). In this case, the size of the overlay image 823b on the display screen 811 is adjusted to be large to take into account the long distance. In this situation, a magnification factor M1 is applied, and the indicated coordinates and the size of the overlay image are automatically adjusted based on the data exchanged between the controller and the display device. This allows the user to easily visually recognize the position even from a long distance, providing comfortable operability.

[0100] 8D shows a scene in which the magnification of the overlay image 823c is changed using the operation ring 851 of the controller 850 while maintaining the distance D2. In this scene, the magnification is adjusted from M1 to M2 by rotating the operation ring 851, and the overlay image 823c is enlarged in greater detail. This adjustment allows the user to enlarge and check a specific portion of the image. Because the magnification is changed in real time based on the rotation of the operation ring, the user can perform the operation intuitively while receiving visual feedback.

[0101] 8A to 8D, the pointing position changes in response to the movement and rotation of the controller, and the size and magnification of the overlay image are adjusted in response to the distance or rotation of the operation ring. This configuration provides the user with a dynamic and interactive operation experience, realizing intuitive operability. This design also maximizes the functionality of the display device for entertainment, education, presentations, and other applications.

[0102] A non-limiting example of the calculation method disclosed in the present disclosure will be described below. In this embodiment, a processing flowchart that integrates two calculation processes (calculation of the position of the pointer cursor on the screen and correction processing of the distance sensor) is shown in Figure 9. These processes can also be performed separately.

[0103] Calculation Process Example 1: Calculation of the Position of a Pointer Cursor on a Screen A description will be given of a communication and processing method between a controller (pointing device) and a display device according to one embodiment. In this process, the controller wirelessly communicates with the display device (S912) and calculates the position of the pointer cursor based on the position pointed by the controller.

[0104] First, acceleration and angular velocity values ​​are acquired by sensors (e.g., acceleration sensor and angular velocity sensor) installed in the controller (S911), and the sensor data is transmitted to the display device via wireless communication (S912). The display device calculates a unit quaternion based on this data (S921). Note that if the sensor directly outputs a unit quaternion, this value can be used as is. A known filter, such as a Madgwick filter, can be used to calculate the unit quaternion.

[0105] Next, a rotation matrix is ​​calculated from the unit quaternion (S922). When the unit quaternion is Q(qr, qi, qj, qk), the rotation matrix C is expressed by the following formula:

[0106] Furthermore, the rotation angles (roll (φ), pitch (θ), yaw (ψ)) of the pointing device are calculated based on this rotation matrix. These rotation angles are calculated using the following formulas: These rotation angles specify the orientation of the pointing device.

[0107] Furthermore, the position coordinates of the pointer cursor are calculated based on the distance (d) from the display device, the screen size (width (W), height (H)), and the number of pixels (horizontal (Ph), vertical (Pv)) (S941). First, the number of pixels per unit length is calculated using the following formula:

[0108] Next, taking into consideration the correction according to the distance (d), the differential distance (dx, dy) of the pointer cursor is calculated by the following formula: By adding this difference distance to the coordinates with the center of the screen as the origin, the final coordinates (x, y) of the pointer cursor are determined (S942).

[0109] These processes are executed repeatedly in real time. The distance d is acquired by a distance sensor inside the controller, and if the value deviates significantly, it is appropriately corrected. This configuration allows the cursor to move intuitively and smoothly according to the position pointed at by the controller, providing the user with a natural operating experience.

[0110] Calculation Processing Example 2: Correction of Distance Sensor Shake Using Acceleration In this example, a method for correcting the shake of the distance sensor measurement value using the output of the acceleration sensor is described. This method improves the accuracy of distance measurement between the pointing device and the display device, enabling more stable operation.

[0111] First, the value of the acceleration sensor installed in the pointing device is sent to the computer. The acceleration sensor measures the acceleration value of the axis along the display direction (hereinafter referred to as the "display axis"), and changes in this value are used as the criterion for the correction process described below.

[0112] The change in the acceleration value of the display axis obtained from the acceleration sensor is evaluated based on a statistical index (e.g., variance) or a set threshold (S931). Depending on the evaluation result, the measurement value of the distance sensor is processed as follows: 1. If the acceleration change of the display axis is large: If the acceleration along the display axis is changing significantly, it is highly likely that the pointing device is moving significantly. In this case, the distance sensor value is used as is without being corrected (S932). 2. If the acceleration change of the display axis is small: If the acceleration change along the display axis is small, it is determined that the pointing device is stable. In this case, the distance sensor value is corrected to suppress time-series changes by smoothing processing or the like (S933). This correction reduces blurring in the distance measurement and improves the reliability of determining the position of the pointer cursor.

[0113] This flowchart (FIG. 9) shows an example in which Calculation Processing Example 1 and Calculation Processing Example 2 are integrated, but it can also be applied when each process is executed independently. This flexibility allows for a variety of applications and enables advanced system control that improves the user experience.

[0114] The present disclosure includes the following embodiments: A001. A method for wirelessly controlling an image on a display using a controller, comprising: (b0) wirelessly acquiring from the controller attitude and movement data of the controller and relative position data with respect to the display; (c0) determining a pointed position in a display coordinate system based on the attitude and movement data and the relative position data; (d0) generating an overlay image; and (e0) displaying the generated overlay image at the pointed position on the display. B001. (A0) a display screen for displaying an image; (B0) a communication unit for wirelessly receiving, from the controller, attitude and movement data of the controller and relative position data of the controller with respect to the display device; (C0) a processor configured to: (i) identify a designated position in a display coordinate system based on the received attitude and movement data and relative position data; and (ii) generate an overlay image; and (D0) a display control unit configured to display the overlay image at the designated position on the display. C001. A controller for controlling image output of a display device, comprising: (1) an input interface for detecting a user operation; (2) a sensor module for generating attitude and movement data and relative position data of the controller with respect to the display device based on the user operation detected by the input interface; and (3) a communication unit for wirelessly transmitting the generated attitude and movement data and relative position data to the display device.A101. A method for wirelessly controlling an image on a display using a controller, comprising: (a) displaying a base image on the display; (b) wirelessly acquiring from the controller attitude and movement data of the controller and relative position data with respect to the display; (c) determining a pointed position within a display coordinate system based on the attitude and movement data and the relative position data; (d) generating an overlay image, determining a size of the overlay image based on the relative position data of the controller, and further enlarging a partial image of the base image based on the pointed position, and generating the overlay image including the enlarged partial image; and (e) superimposing the generated overlay image on the pointed position of the base image on the display. B101. (B) a communication unit that wirelessly receives, from the controller, attitude and movement data of the controller and relative position data of the controller with respect to the display device; (C) a processor configured to: (i) identify a designated position in a display coordinate system based on the received attitude and movement data and relative position data of the controller; and (ii) determine a size of the overlay image based on the relative position data, and enlarge a portion of a substrate image corresponding to the designated position to generate an overlay image; and (D) a display control unit configured to display the generated overlay image superimposed on the designated position on the substrate image. C011. The controller of C001 or any embodiment, wherein the sensor module comprises: an attitude and movement measurement module that measures attitude and movement data of the controller; and a distance sensor module that measures the relative position of the controller with respect to the display device.C012. The controller according to C011 or any of the embodiments, wherein the attitude and movement measurement module includes a 9-axis sensor. C013. The controller according to C011 or any of the embodiments, wherein the distance measurement module includes a laser distance sensor. C021. The controller according to C001 or any of the embodiments, wherein the controller has an elongated shape and is configured to be used with its longitudinal direction facing the display device. C022. The controller according to C021 or any of the embodiments, wherein the controller has a cylindrical outer shape, is hollow in the longitudinal direction, and is open at both ends. C023. The controller according to C022 or any of the embodiments, further comprising an operation ring on the outer periphery of the cylindrical controller, configured to control the operation of the display device by rotation. C031. A controller according to C001 or any other embodiment, wherein the input interface further comprises a position reset button configured to, when operated, display a target (reset) cursor on the screen of the display device and reset the initial position of the controller. A011. A method according to A001 or any other embodiment, wherein (b) comprises: (b0) acquiring initial attitude and movement data of the controller and initial relative position data with respect to the display; and (c) comprises: (c0) determining an initial pointing position within a display coordinate system based on the initial attitude and movement data and the initial relative position data. A012. A method according to A011 or any other embodiment, wherein (b) further comprises displaying a target (reset) cursor on the display and requesting a user to hold the controller aimed at the target (reset) cursor.A013. The method according to A012 or any of the embodiments, wherein (b) comprises: acquiring the attitude and movement data and the relative position data of the controller in a state where the controller is held pointing at the target (reset) cursor as the initial attitude and movement data and the initial relative position data. A015. A method according to A011 or any of the embodiments, wherein (b) includes: (b1) acquiring current attitude and movement data of the controller and current relative position data with respect to the display in real time after determining the initial pointing position; (c) includes: (c1) determining a current pointing position in a display coordinate system based on the initial attitude and movement data, the initial relative position data, the current attitude and movement data, and the current relative position data; (d) includes: (d1) generating an overlay image, determining a size of the overlay image based on the current relative position data of the controller, and further enlarging a partial image of the base image based on the current pointing position, and generating the overlay image including the enlarged partial image; and (e) includes: (e0) displaying the generated overlay image superimposed on the pointing position of the base image on the display in real time.

[0115] In another embodiment, the present disclosure includes a method for wirelessly controlling an image on a display using a controller, the method comprising: detecting a distance between the controller and a predetermined area of ​​the image on the display; and zooming in (magnification increase) or zooming out (magnification decrease) the image of the predetermined area in accordance with the detected distance. Zooming in and out may be performed using the above-described overlay processing or a GUI (graphical user interface) capable of zooming. Alternatively, a method may be used in which, when scanning an image, only the predetermined area (e.g., an overlay image including a designated position) is displayed at a magnification corresponding to the detected distance. The predetermined area corresponds to, for example, the circular area of ​​the overlay image 823a in FIG. 8B . The magnification may be changed in accordance with the distance detected by a distance sensor built into the controller in addition to information from the operation ring. Furthermore, when a user points the pointer of the controller at the display and captures an image of the predetermined area, the captured image may be displayed on the display or transferred to an external electronic device.

[0116] In yet another embodiment, the present disclosure includes a controller or pointing device for controlling image output of a display device, the controller or pointing device comprising: (1) an input interface for detecting user operations; (2) a sensor module for detecting the distance between the controller and a predetermined area of ​​the display; and (3) a communication unit for wirelessly transmitting data to the display device for zooming in (enlarging) or zooming out (reducing) an image of the predetermined area depending on the distance detected by the sensor module.

[0117] The present disclosure includes, as another embodiment, a display device wirelessly controlled by a controller, comprising: (1) a display screen that displays an image; (2) the controller having a sensor module that detects the distance between the controller and a predetermined area of ​​the display device, and a communication unit that wirelessly receives the distance detected by the sensor module from the controller; and (3) a display control unit that zooms in (enlarges) or zooms out (reduces) an image of the predetermined area depending on the distance detected by the sensor module and transmitted from the communication unit.

[0118] The present disclosure also provides, as another embodiment, an imaging system comprising a display device and a controller for wirelessly controlling image output of the display device, wherein (1) the controller comprises: an input interface for detecting user operations; a sensor module for detecting the distance between the controller and a predetermined area of ​​the display; and a communication unit for transmitting distance data detected by the sensor module to the display device; and (2) the display device comprises: an interface for receiving the distance detected by the sensor module from the communication unit; and a calculation module for zooming in (enlarging) or zooming out (reducing) an image of the predetermined area according to the detected distance received by the interface.

[0119] The present disclosure also includes embodiments and examples illustrated in Figures 10A to 10P. [Summary of the Invention] (Advantages, Effects, Differences from the Prior Art, and Fields of Applicability) This invention relates to a sensor-equipped pointing device (remote controller) that functions as a remote controller for a display. This device achieves high-precision pointing operations through a combination of an integrated acceleration sensor, an angular velocity sensor, and a distance sensor. The distance sensor in the device measures the distance between the device and the display, and the angular velocity sensor detects the angle of the device. Based on this information, pointing operations corresponding to pixel coordinates on the display are possible. Furthermore, operability is improved by using an acceleration sensor to detect forward and backward movement of the device and adjusting the sensitivity of the distance sensor. An advantage of this invention is that it enables display operation with higher precision than conventional remote controllers. Furthermore, spatial coordinate correction using a group of sensors provides a more intuitive and smooth operation experience. This technology is expected to be applied in a variety of fields, including television, presentations, and interactive exhibits. Compared to conventional remote controllers, this invention offers superior advantages in both operability and accuracy, and is believed to significantly contribute to improving user interfaces. [Prior Technology and Its Disadvantages] Conventional pointing operations required a sensor mechanism on both the display and the remote control device. For example, the Nintendo Wii Remote used a sensor bar attached to the TV. VR devices also use a camera sensor on the device itself and an infrared LED on the controller. It was difficult to achieve accurate pointing operations on a display using a pointing device alone. Furthermore, with conventional simple pointing devices that only use acceleration and angular velocity, the degree of on-screen movement in response to tilting the pointing device remained constant even when the distance between the device and the display changed.The coordinates on the screen indicated by pointing respond less to changes in angle when the pointing device is close to the screen, and more strongly when the pointing device is farther away, creating a natural experience for the user. With conventional devices, coordinate changes due to pointing operations are uncorrelated with the distance from the screen, creating a sense of discomfort for users. This poses a major challenge in device operability and user experience. This invention achieves pointing operations that correlate with distance using a single pointing device. Because sensors or markers are not required on the receiving side, fewer components are required, reducing manufacturing costs. [Method of Implementation - Invention 1] Figure 10A shows an overview of this disclosure. Figure 10B shows the internal block diagram of the pointing device. Figure 10C shows the internal block diagram of the display. Figure 10D shows the overall processing flow. 10.1 is the pointing device, and 10.2 is the display with a built-in computer. Figure 10B shows the internal block diagram of the pointing device, and Figure 10C shows a block diagram of the display. The overall processing flow is shown in Figure 10D, and the detailed steps are as follows: 1. Connect the pointing device (10.1) to a computer (10.2) or the like via wireless or other means. 2. Calculate a unit quaternion from the acceleration and angular velocity values ​​of the sensor in the pointing device, or use a unit quaternion if it is output from the sensor. There are several well-known methods for calculating a unit quaternion from the angular velocity and acceleration of the sensor, such as the Madgwick filter, and a Madgwick filter can be used. 3. Calculate a rotation matrix from the unit quaternion. Convert the unit quaternion into Q(q. r , q i , q j , q k ) then the rotation matrix C can be calculated using the following formula: 4. Convert the quaternion rotation matrix into the rotation angles of the pointing device (roll φ, pitch θ, yaw ψ). (A flow diagram of the entire signal flow is provided.) The conversion formula is as follows: 5. Rotation angle, distance d between the pointing device and the display, screen size (W, H), number of pixels (Ph , P v ) to calculate the differential distance from the origin (d x ,d y ) is calculated. When calculating the difference distance, the distance d is proportionally correlated to solve the problem of incongruity. First, the number of pixels per unit length (p x , p y ) is p x = P h / W (1) p y = P v / H (2) From these, the differential distance can be calculated as follows: d x = p x * d * tan(ψ) (3) d y = p ｙ * d * tan(θ) (4) 6. In this case, the distance d is obtained from the distance sensor built into the pointing device. If the value is significantly unstable, the value is corrected (the correction method is shown below). 7. The center of the screen is set as the origin (0,0), and (d x ,d y) is used as the pointer cursor coordinates (x,y) indicated by the pointing device. 8. These processes are repeated (A flow diagram of the overall signal flow is provided.) The method of correcting for distance sensor shake using acceleration is as follows (A flow diagram of the overall signal flow is provided.) 1. The value of the acceleration sensor in the pointing device is sent to the computer. 2. The degree of change in the value of the acceleration sensor's axis in the display direction is determined using variance or a threshold value. 3. If the change in acceleration in the display direction is large, the distance sensor value is not corrected. 4. If the change in acceleration in the display direction is small, the distance sensor value is corrected to reduce time-series changes by smoothing or other processing. (Use diagrams and flow diagrams to explain how the display works.) [Advanced Ideas] Based on the above invention, the following developments are possible. Invention 2. Support for multiple displays, Fig. 10E It is also possible to operate multiple displays with a single pointing device. If the relative positions of multiple displays are known to each other, or to the computer of the reference display, the distance and angle can be calculated, and the pointer can be moved across the displays. Invention 3. Zooming the drawing range, Fig. 10F, 10G. The pointing device has a zoom button, and by operating it, the drawing can be enlarged or reduced based on the pointer coordinates displayed on the display. The enlarged display range has a circular shape, and its radius and area expand or contract according to the distance d. (This provides an experience similar to that of a telescope or a zoom lens on a camera.)

[0120] Supplementary Item 1: A pointing device for controlling an image on an external display, the pointing device having: a sensor for detecting movement of the pointing device away from the external display; a unit for compressing three-dimensional movement of a pointer of the pointing device as the pointing device moves away from the external display in response to a signal from the sensor; and a correction unit for correcting movement due to displacement of the pointer at long distances. Supplementary Item 2: The pointing device according to Supplementary Item 1, wherein the correction unit is configured to prevent overly sensitive movement due to displacement of the pointing device at long distances. Supplementary Item 3: A pointing device for controlling an image on an external display, the pointing device having: a distance sensor for measuring the distance between the pointing device and the external display; an angular velocity sensor for detecting the angle of the pointing device; and an acceleration sensor for detecting forward and backward movement of the pointing device and adjusting the sensitivity of the distance sensor. Supplementary Item 4: The pointing device according to Supplementary Item 3, characterized in that it enables a pointing operation corresponding to pixel coordinates on the display based on detection signals from the distance sensor, the angular velocity sensor, and the acceleration sensor. Supplementary Item 5: The pointing device according to any one of Supplementary Items 1 to 4, wherein the external display and the pointing device each have a transmitting and receiving antenna and transmit and receive signals wirelessly. Supplementary Item 6: An external display having a unit that receives a sensor signal from the pointing device according to any one of Supplementary Items 1 to 4, and a computer that calculates a rotation angle by calculating and acquiring a unit quaternion based on data received from the receiving unit, and calculates a differential distance from the rotation angle, distance, screen size, and number of pixels. Supplementary Item 7: A display system having a plurality of external displays according to Supplementary Item 6, wherein the pointing device controls images across the plurality of external displays.Supplementary Item 8: A display system comprising the pointing device according to any one of Supplementary Items 1 to 4 having a zoom button, and a unit that operates the zoom button to enlarge or reduce a drawing based on pointer coordinates displayed on the external display.Supplementary Item 9: A method for controlling an image on the external display using the pointing device according to any one of Supplementary Items 1 to 4, comprising the steps of: detecting movement of the pointing device away from the external display by the sensor; compressing three-dimensional movement of the pointer of the pointing device as it moves away from the external display based on the detection signal from the sensor; and correcting movement due to displacement of the pointer at long distances.Supplementary Item 10: A method for controlling an image on the external display using the pointing device according to any one of Supplementary Items 1 to 4, comprising the steps of: measuring the distance between the pointing device and the external display by the distance sensor; detecting the angle of the pointing device by the angular velocity sensor; and detecting forward and backward movement of the pointing device by the acceleration sensor and adjusting the sensitivity of the distance sensor.

[0121] Terms and expressions used in the present specification and claims should be interpreted openly and not restrictively unless otherwise specified. For example, terms such as "comprises" and "includes" mean "including without limitation" or "including but not limited to" and do not exclude additional elements and unrecited configurations. Expressions such as "have" should be interpreted as meaning "have at least."

[0122] When multiple elements and configurations listed herein are connected by "and," it is understood to mean not only a configuration including all of them, but also a configuration including any one or more of them. Similarly, when connected by "or," it is understood to mean a configuration including any one, any more, or all of them.

[0123] As used herein, singular terms may be construed to include plurals where appropriate. It should also be considered that plural terms may include the singular where appropriate. Words such as "one," "an," and the like do not exclude the possibility of including plural references unless specifically stated otherwise.

[0124] The embodiments and examples disclosed herein are presented as examples of applicable configurations of the invention, and are not intended to be limiting. Multiple embodiments can be used independently and can also be combined with each other. Features or components of different embodiments can be combined in part or in whole.

[0125] While preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. It is not intended that the present invention be limited to the specific examples provided within the specification. Furthermore, the present invention is not limited to the specific depictions, configurations, or proportions set forth in the illustrations, as such may be adapted to a variety of conditions and variables. The following claims define the scope of the present invention, and it is intended that methods and structures within the scope of these claims, and their equivalents, be covered.

Claims

1. 1. A method for wirelessly controlling an image on a display using a controller, comprising: (b0) wirelessly acquiring from the controller attitude and movement data and relative position data of the controller with respect to the display; (c0) determining a pointing position in a display coordinate system based on the posture and movement data and the relative position data; (d0) generating an overlay image; (e0) displaying the generated overlay image at the indicated position on the display; and (f0) resetting the overlay image to a predetermined origin position in response to detecting a reset operation, and automatically recalibrating an offset value of a motion sensor in the controller simultaneously with resetting the overlay image; A method for providing the above.

2. 1. A method for wirelessly controlling an image on a display using a controller, comprising: (a) displaying a substrate image on said display; (b) wirelessly acquiring from the controller attitude and movement data and relative position data of the controller with respect to the display; (c) determining a pointing position within a display coordinate system based on the posture and movement data and the relative position data; (d) generating an overlay image, determining a size of the overlay image based on the relative position data of the controller, and further enlarging a partial image of the base image based on the designated position, and generating the overlay image including the enlarged partial image; and (e) displaying the generated overlay image superimposed on the base image at the designated position on the display; and (f) temporarily reducing the XY axis motion detection sensitivity during the zoom execution period; A method for providing the above.

3. 10. The method of claim 1, The (b) is (b0) acquiring initial attitude and movement data of the controller and initial relative position data with respect to the display; The (c) is (c0) determining an initial pointing position in a display coordinate system based on the initial posture and movement data and the initial relative position data; method.

4. 4. The method of claim 3, The (b) is displaying a reset cursor on the display and requesting a user to hold the controller toward the target cursor. method.

5. 5. The method of claim 4, The (b) is acquiring, as the initial posture and movement data and the initial relative position data, the posture and movement data and the relative position data of the controller in a state in which the controller is held facing the reset cursor; Equipped with method.

6. 4. The method of claim 3, The (b) is (b1) after determining the initial pointing position, acquiring current attitude and movement data of the controller and current relative position data with respect to the display in real time; The (c) is (c1) determining a current pointing position in a display coordinate system based on the initial attitude and movement data, the initial relative position data, the current attitude and movement data, and the current relative position data; The (d) is (d1) generating an overlay image, determining a size of the overlay image based on the current relative position data of the controller, and further enlarging a partial image of the base image based on the current pointing position, and generating the overlay image including the enlarged partial image; The (e) is (e0) displaying the generated overlay image superimposed on the designated position of the base image on the display in real time; method.

7. A display device wirelessly controlled by a controller, (A0) a display screen for displaying an image; (B0) a communication unit that wirelessly receives, from the controller, attitude and movement data of the controller and relative position data of the controller with respect to the display device; (C0) a processor, (i) determining a pointing position within a display coordinate system based on the received attitude and motion data and relative position data; and (ii) generating an overlay image to be displayed at the indicated location; a processor configured to execute the (D0) a display control unit configured to display the overlay image at the indicated position on the display; Equipped with A display device configured to reset the overlay image in response to detecting the reset operation, and to automatically recalibrate an offset value of a motion sensor in the controller simultaneously with the reset.

8. 8. The display device according to claim 7, The processor: determining a size of the overlay image based on the relative position data, and enlarging a portion of the base image corresponding to the indicated position to generate the overlay image; configured to run The display control unit The generated overlay image is displayed superimposed on the designated position of the base image. Display device.

9. A controller for controlling image output of a display device, comprising: (1) an input interface for detecting user operations; (2) a sensor module that generates attitude and motion data and relative position data of the controller with respect to the display device based on user operations detected by the input interface; and (3) a communication unit that wirelessly transmits the generated attitude and motion data and relative position data to the display device; Equipped with The sensor module further comprises a memory unit that stores an offset value of the controller, the controller being configured to automatically recalibrate the offset value in response to detecting a reset operation by the input interface.

10. 10. The controller of claim 9, The sensor module includes: an attitude and motion measurement module that measures attitude and motion data of the controller; and a distance sensor module for measuring the relative position of the controller with respect to the display device; Equipped with controller.

11. 11. The controller of claim 10, The posture and movement measurement module includes a 9-axis sensor. controller.

12. 11. The controller of claim 10, the distance measurement module includes a laser distance sensor; controller.

13. 10. The controller of claim 9, the controller has an elongated shape and is configured to be used with its longitudinal direction facing the display device; The device further has a function of automatically switching from a pointing mode to a gesture recognition mode in response to a specific gesture input using the elongated cylindrical outer shape. controller.

14. 14. The controller of claim 13, the controller has a cylindrical outer shape, is hollow in the longitudinal direction, and is open at both ends; The controller further comprises a function of automatically switching an operation mode in response to a gesture input using the hollow structure.

15. 15. The controller of claim 14, an operation ring configured to control an operation of the display device by a rotation operation, the operation ring being disposed on an outer periphery of the cylindrical controller; The operation ring further includes a touch sensor that detects the movement of the fingers of a hand holding the controller in addition to the rotational operation, and is configured to automatically switch the operation mode based on the detection of the touch sensor. controller.

16. 11. The controller of claim 10, the input interface further includes a position reset button configured to, when operated, cause a reset cursor to be displayed on the screen of the display device and reset the initial position of the controller; the position reset button is configured to automatically recalibrate an offset value of a motion sensor in the controller simultaneously with resetting the initial position; controller.

17. 1. A method for wirelessly controlling an image on a display using a controller, comprising: detecting a distance between the controller and a predetermined area of ​​the display; Zooming in or out of the image of the predetermined area according to the detected distance; and Temporarily reducing the XY axis motion detection sensitivity while zooming in or out: A method for providing the above.

18. A controller for controlling image output of a display device, comprising: (1) an input interface for detecting user operations; (2) a sensor module that detects the distance between the controller and a predetermined area of ​​the display; (3) a communication unit that wirelessly transmits to the display device data for zooming in or zooming out the image of the predetermined area according to the distance detected by the sensor module; Equipped with The controller is configured to temporarily reduce XY axis motion detection sensitivity when transmitting data to zoom in or zoom out depending on the distance detected by the sensor module.

19. A display device wirelessly controlled by a controller, (1) a display screen for displaying images; (2) the controller includes a sensor module for detecting a distance between the controller and a predetermined area of ​​the display device, and a communication unit for wirelessly receiving the distance detected by the sensor module from the controller; (3) a display control unit that displays an image of the predetermined area by zooming in or zooming out in accordance with the distance detected by the sensor module and transmitted from the communication unit; Equipped with a display device configured to transmit to the controller a signal for temporarily lowering the sensitivity of detecting motion along the X and Y axes during a period in which the zoom-in or zoom-out display is performed in accordance with the distance detection;

20. 1. An image system comprising a display device and a controller for wirelessly controlling an image output of the display device, (1) The controller an input interface for detecting a user operation; a sensor module for detecting a distance between the controller and a predetermined area of ​​the display; a communication unit for transmitting distance data detected by the sensor module to the display device; Equipped with (2) The display device is an interface for receiving the distance detected by the sensor module from the communication unit; a calculation module that zooms in or zooms out an image of the predetermined area according to the detected distance received through the interface; the calculation module is configured to execute control to temporarily reduce the XY axis motion detection sensitivity of the controller during a period in which zoom-in or zoom-out is performed; Imaging system.