Responsive display

Abstract

To provide an improved method for generating, by a computing device, graphical elements on a display.SOLUTION: A computing device comprises at least part of one or more graphical elements 420, 430, 440, and identifies an obstructed portion of a display 110 that is blocked by objects 435, 415. The computing device identifies an unobstructed portion of the display 110 that is not blocked by the objects 435, 415, and arranges the one or more graphical elements 430, 420, 440 from the obstructed portion to the unobstructed portion.SELECTED DRAWING: Figure 4B

Description

[Background technology]

[0001]

[0001] Most non-mobile electronic displays, both large and small, are mounted in a fixed location or held in a larger housing. These stationary displays are more susceptible to being blocked by other stationary objects in front of the display or by moving objects moving in front of the display.

[0002]

[0002] Aspects of the present disclosure are illustrated by way of example and not by way of limitation in the accompanying drawings in which like reference numerals indicate like elements and in which: [Brief description of the drawings]

[0003] [Figure 1A] FIG. 1A is a schematic diagram of a display including the field of view of a viewpoint T across the width of the display. [Figure 1B] FIG. 1B is a schematic diagram of a display including the field of view of a viewpoint T relative to the height of the display. [Figure 2A]

[0004] FIG. 2A is a schematic diagram of a display including a graphical element that is occluded by an object. [Figure 2B]

[0005] FIG. 2B is a schematic diagram of a display having graphical elements repositioned to an unobstructed portion of the display screen, in accordance with one or more embodiments of the present disclosure. [Figure 3A]

[0006] FIG. 3A is a schematic diagram of a display including a graphical element that is occluded by multiple objects. [Figure 3B]

[0007] FIG. 3B is a schematic diagram of a display having graphical elements repositioned to an unobstructed portion of the display screen, in accordance with one or more embodiments of the present disclosure. [Figure 4A]

[0008] FIG. 4A is a schematic diagram of a point-of-sale kiosk that includes a display having graphical elements that are obstructed by multiple objects. [Figure 4B]

[0009] FIG. 4B is a schematic diagram of a point-of-sale kiosk including a display having graphical elements repositioned to an unobstructed portion of the display screen in accordance with one or more embodiments of the present disclosure. [Figure 5A]

[0010] FIG. 5A is a schematic diagram of an electronic billboard display having a graphical element that is occluded by multiple objects. [Figure 5B]

[0011] FIG. 5B is a schematic diagram of an electronic billboard display having graphical elements repositioned to an unobstructed portion of the display screen in accordance with one or more embodiments of the present disclosure. [Figure 6A] FIG. 6A is a schematic diagram of a vehicle display having a graphical element that is obstructed by an object. [Figure 6B] FIG. 6B is a schematic diagram of a vehicle display having graphical elements repositioned to an unobstructed portion of the display screen in accordance with one or more embodiments of the present disclosure. [Figure 7] FIG. 7 is a flowchart illustrating a method implemented by a computing device in accordance with one or more embodiments of the present disclosure. [Figure 8] FIG. 8 is a schematic block diagram illustrating an exemplary computing device in accordance with one or more embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004]

[0005] Generally speaking, embodiments of the present disclosure improve upon conventional display techniques by relocating on-screen elements that are blocked from view to other locations on the screen.

[0005]

[0006] Often, images on a display are blocked by stationary objects or objects that move across the display screen. For example, a display housed within a self-checkout kiosk may be blocked by items placed on the kiosk's scale. In this case, a viewer may not be able to see all of the image produced on the display.

[0006]

[0007] Conventional techniques for modifying a display to avoid being blocked include setting a background image on a window or setting a projector to display above something. That is, previous techniques for modifying a display include repositioning the projected image or the entire display. These solutions are not suitable for stationary or large displays. Unlike mobile devices or displays mounted on rotatable brackets, in stationary displays, the blocked view caused by items in front of the display is a fundamental problem that cannot be solved by repositioning the display.

[0007]

[0008] Conventional techniques also do not dynamically adjust the display based on environmental stimuli, such as captured images, weight, and / or motion detection.

[0008]

[0009] Solutions provided include a display that automatically adjusts an active display displaying graphical elements in a responsive manner. The adjustments can be based on the location and other characteristics of an object blocking the display. The location and other characteristics can be sensed by one or more sensors and software techniques including, for example, computer vision, object detection, infrared lasers, scanners, and scales.

[0009]

[0010] FIG. 1A shows a top view of the display 110 with a field of view (FOV) for the display 110. The FOV is determined by the location of a fixed point, i.e., a viewpoint T. The FOV may refer to coverage of an entire area, rather than a single fixed focal point. The FOV also describes the angle Θ at which a person can see the visible world from the viewpoint T. The display FOV may also refer to the range of angles at which the image displayed on the monitor remains suitable for the user. The display FOV may be measured from the normal axis of the display to each of the four directions (i.e., up, right, left, and down). The viewpoint T is not limited to a single fixed point. More than one viewpoint position may be used to determine more than one FOV.

[0010]

[0011] As shown in FIG. 1A, a viewpoint T of a display 110 is blocked by one or more objects A and B located in the FOV with respect to the width of the display 110. Object A blocks the target viewpoint T in the width direction. Object B partially blocks viewpoint T. Both objects A and B are located on the left side of the display. Meanwhile, object C is behind viewpoint T and does not block viewpoint T in the field of view. For the same example, FIG. 1B shows a side view of the display 110 with the FOV for the display 110 with respect to the height. Objects A and B completely or partially block the viewpoint, while object C does not block the viewpoint in the field of view. Objects A and B are located in the lower half of the display screen. Again, object C is behind viewpoint T. In this case, the display will be modified to move screen elements to the upper right side of the display to avoid objects A and B located on the lower left side of the screen. Object C is not considered because it does not block viewpoint T and is not in the FOV.

[0011]

[0012] 1A and 1B, the FOV and angle Θ are determined by a given location of the viewpoint T. The position of the viewpoint T may be determined according to the size and quality of the display and the maximum distance from the display to the viewpoint T at which the quality of the image from the viewpoint T remains suitable for the user. For example, for a smaller display, the viewpoint T will have a shorter distance from the display compared to a larger display.

[0012]

[0013] 2A is a schematic diagram 200a of a display 110 from a forward-facing perspective. In this example, the display 110 includes graphical elements 215, 220, and 225. An object 230, centered in front of the screen, blocks the graphical element 220. The object 230 may be, for example, a person or a commercial item. One or more people may be unable to see portions of the display 110 that are behind the object 230.

[0013]

[0014] An embodiment of the present disclosure provides a method for detecting a view of the display 110 that is partially blocked by an object and relocating one or more graphical elements from the blocked portion of the display to an unblocked portion of the display.

[0014]

[0015] Figure 2b shows a schematic diagram 200b in which the previously blocked graphical element 220 referenced in Figure 2a is now unblocked, in which the unblocked graphical element objects 215 and 225 remain in their positions while the graphical element 220 is moved to an unobstructed position above the object 230.

[0015]

[0016] In this example, graphical elements 215, 220, and 225 are the same size. Because object 230 blocks only graphical element 220, graphical element 220 can be repositioned to a new, unobstructed position without moving graphical elements 215 and 225.

[0016]

[0017] The display devices disclosed herein may be coupled to or may include any one or more sensors 120, including, for example, image sensors, weight sensors, and scatter scanners. The sensors 120 may be used to detect, identify, and locate objects in front of the display.

[0017]

[0018] In other instances where the graphical elements do not fit in the unobstructed area, the graphical elements may be resized to fit the unobstructed display area without compromising the integrity of the image.

[0018]

[0019] Figure 3B is a schematic diagram 300b of the display 110 adjusted in response to objects 230 and 310 shown in schematic diagram 300a of Figure 3A being placed in front of the display 110. In Figure 3B, the graphical elements 215, 220 and 225 have been relocated to an unblocked or unobstructed portion of the display screen. In this figure, the graphical elements 215, 220 and 225 have been repositioned to the unobstructed portion of the display. Because the unobstructed portion of the display is smaller, the graphical objects 215, 220 and 225 have been resized to fit into this smaller unobstructed viewing area.

[0019]

[0020] 4A is a schematic diagram of a point-of-sale kiosk 400a that includes a display 110 and a base 410. During a sales transaction, sale items 435, 415 are placed on the kiosk base 410. As the items are placed on the base 410, the identity of each object is determined and displayed on the display 110.

[0020]

[0021] 4A, computer vision based self-checkout kiosks that utilize a display as their background may have their view of graphical elements 430 and 420 blocked by items 435, 415 placed on a platform 410. Moving items to touch (or view) the display can be problematic because items must be spaced apart on a scanning area of ​​limited size.

[0021]

[0022] Figure 4B shows a solution to the problem shown in Figure 4A. In Figure 4B, diagram 400b shows graphical elements 430 and 440 being repositioned to an unobstructed area above objects 415 and 435.

[0022]

[0023] Weight sensors installed at the platform per 410 can be used to determine the weight, location, and size of each item. Obstructed and unobstructed areas can be determined using the weight sensor data.

[0023] Billboard

[0024] In other embodiments, electronic signage such as billboards displaying advertisements at retail locations or on the street, for example at bus stops, may be configured to implement the methods disclosed herein.

[0024]

[0025] 5A is a schematic diagram of an electronic billboard display 110 having graphical elements 510, 515, 520, 530, and 550. In this example, the graphical elements 510, 515, 520, 530, and 550 may be distributed across the entire display 110 since there are no objects blocking the display. For advertising purposes, additional props or sale items may be placed in front of the display. In this case, the area of ​​the display may be reduced according to the size of the props placed in front of the display.

[0025]

[0026] 5B is a schematic diagram of an electronic billboard display in which graphical elements 510, 515, 520, 530, and 550 have been repositioned to an unobstructed portion of display 110 according to the location of a prop placed in front of the display. In this example, a banana stand 560 has been placed in front of the display. The graphical elements have now been repositioned above banana stand 560.

[0026]

[0027] In some examples, the graphical elements may be prioritized according to the importance of the graphical elements 510, 515, 520, 530, and 550. For example, the word banana 515 and the price of bananas per pound may have a higher priority than an image of a banana and sale text 550. If the unobstructed area is not large enough to contain all the graphical elements in the display, the lower priority graphical elements may be omitted or resized to a smaller size to fit on the display. The priority of the graphical elements may be determined by the creator of the digital advertisement.

[0027]

[0028] Figure 6A is a schematic diagram 600a of a vehicle 605 having a display 110 located within the vehicle. The display includes a graphical element 620 that is obstructed by an object 610. The object 610 is positioned such that the view is partially blocked for the driver of the vehicle. A sensor 120 is capable of detecting a cup in a cup holder in front of the display. The sensor 120 may be a camera or a scanner. Once detected, the display element 620 is repositioned to the top of the screen in an area where the cup does not obstruct the driver's view of the screen, as shown in schematic diagram 600b of Figure 6B.

[0028]

[0029] 7 is a flow chart illustrating a method (700) performed by a computing device. The method (700) includes detecting (710) that a view of a display is partially blocked by an object. The display includes at least a portion of one or more graphical elements. The method further includes, in response to identifying (720) an occluded portion of the display that is blocked by the object, the computing device being configured to identify (730) an unoccluded portion of the display that is not blocked by the object. The method further includes disposing (740) one or more graphical elements from the occluded portion to the unoccluded portion.

[0029]

[0030] According to some embodiments, the computing device automatically adjusts the graphical elements in a responsive manner based on the size of the item placed on the scanning platter.

[0030]

[0031] According to some embodiments, the method includes determining that the occluded portion includes one or more graphical elements, including comparing a shape and position of the object to the shape and position of each of the graphical elements.

[0031]

[0032] According to some embodiments, the method includes identifying occluded and unoccluded portions, including identifying a shape of the object from an image captured by an image sensor.

[0032]

[0033] According to some embodiments, the method includes detecting that a view is partially blocked, including receiving an object detection indication from a weight sensor in front of the display.

[0033]

[0034] According to some embodiments, the computing device includes a point of sale terminal.

[0034]

[0035] According to some embodiments, the one or more graphical elements are included within a purchasing interface that is at least partially blocked to a user of a point of sale terminal attempting to purchase an object.

[0035]

[0036] According to some embodiments, the computing device comprises an electronic billboard.

[0036]

[0037] According to some embodiments, the display is a smart mirror.

[0037]

[0038] According to some embodiments, the obstructed portion of the display includes an image of a person standing in front of the smart mirror.

[0038]

[0039] According to some embodiments, the computing device is capable of detecting that a view of the display is no longer blocked by the object, and in response, identifies a previously occluded portion of the display that was previously blocked by the object by the computing device.

[0039]

[0040] According to some embodiments, the graphical element is repositioned to a previously occluded portion, which may occur after the previously occluded portion becomes unoccluded, for example, when an object is moved out of the way of the display.

[0040]

[0041] 8 is a schematic block diagram illustrating an exemplary computing device 800 according to one or more embodiments of the present disclosure. The exemplary computing device includes a processing circuit 805, a memory circuit 815, and an interface circuit 835. The processing circuit 805 is communicatively coupled to the memory circuit 815 and the interface circuit 835, for example, via one or more buses 810. The processing circuit 805 may include one or more microprocessors, microcontrollers, hardware circuits, discrete logic circuits, hardware registers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or combinations thereof. For example, the processing circuit 805 may be programmable hardware capable of executing software instructions 820 stored as a machine-readable computer program, for example, in the memory circuit 815. The memory circuitry 815 of various embodiments may include any non-transitory machine-readable medium, whether volatile or non-volatile, known or that may be developed in the art, including, but not limited to, solid-state media (e.g., SRAM, DRAM, DDRAM, ROM, PROM, EPROM, flash memory, solid-state drives, etc.), removable storage devices (e.g., secure digital (SD) cards, miniSD cards, microSD cards, memory sticks, thumb drives, USB flash drives, ROM cartridges, universal media disks), fixed drives (e.g., magnetic hard disk drives), or the like, either in whole or in any combination.

[0041]

[0042] The interface circuitry 835 may be a controller hub configured to control input and output (I / O) data paths of the computing device 800. Such I / O data paths may include a data path for exchanging signals over a communication network and a data path for exchanging signals with a user. For example, the interface circuitry 835 may include a transceiver configured to send and receive communication signals over one or more of a cellular network, an Ethernet network, or an optical network. The interface circuitry 835 may also include (or be communicatively connected to) one or more of a graphics adapter, a display port, a video bus, a touch screen, a graphical processing unit (GPU), a display port, a liquid crystal display (LCD), and a light emitting diode (LED) display for presenting visual information to a user. The interface circuitry 835 may also include one or more of a pointing device (e.g., a mouse, a stylus, a touch pad, a track ball, a pointing stick, a joystick), a touch screen, a microphone for voice input, a light sensor for optical recognition of gestures, and a keyboard for character input. In some embodiments, computing device 800 may additionally or alternatively include, either as part of or communicatively coupled to interface circuitry 835, a combination of one or more displays 110, I / O devices, illumination sources, and / or near-field receivers, as described above, and sensors 120 including one or more cameras 825, weight sensors 850, scanners 860.

[0042]

[0043] The interface circuitry 835 may be implemented as a single physical component, or may be implemented as multiple physical components arranged in series or separately, any of which may communicate with or be communicatively coupled to any of the others via the processing circuitry 805. For example, the interface circuitry 835 may include an output circuit (e.g., a transmitter circuitry 840 configured to send communication signals over a communication network) and an input circuit (e.g., a receiver circuitry 845 configured to receive communication signals over a communication network). Similarly, the output circuitry may include the display 110, while the input circuitry may include a keyboard, a touch screen, or a card reader. Other examples, permutations, and arrangements of the above, and equivalents thereof, will be readily apparent to those of ordinary skill in the art.

[0043]

[0044] Other embodiments include a non-transitory computer-readable medium (e.g., memory circuitry 815) having stored thereon a computer program (e.g., control program 820) that includes software instructions that, when executed on the processing circuitry 805 of the computing device 800, cause the computing device 800 to perform any of the methods disclosed herein.

[0044]

[0045] The present disclosure may, of course, be practiced otherwise than as described herein without departing from the essential characteristics of the disclosure. The present embodiments are to be considered in all respects as illustrative and not restrictive, and all changes that come within the meaning and range of equivalency of the appended claims are intended to be embraced therein. Although the steps of various processes or methods described herein are shown and described in a certain sequence or chronological order, the steps of any such process or method are not limited to be performed in any particular sequence or order unless otherwise indicated. In fact, the steps in such processes or methods may generally be performed in various sequences and orders while still being within the scope of the present disclosure.

Claims

1. Detecting that a view of the display is partially blocked by an object; identifying an occluded portion of the display that is blocked by the object and that includes at least a portion of one or more graphical elements; identifying an unobstructed portion of the display that is not blocked by the object; and disposing the one or more graphical elements from the occluded portion to the unoccluded portion; A method for providing the above.

2. The method of claim 1 , wherein detecting that the view is partially blocked comprises receiving an object detection indication from a weight sensor in front of the display.

3. The method of claim 1 , wherein identifying the occluded and unoccluded portions comprises identifying a shape of the object from an image captured by an image sensor.

4. The method of claim 1 , wherein determining that the occluded portion comprises the one or more graphical elements comprises comparing a shape and position of the object to a shape and position of each of the graphical elements.

5. The method of claim 1 , further comprising positioning the one or more graphical elements upon detecting movement of the object into the occluded portion.

6. the computing device comprises a point of sale terminal; the one or more graphical elements are included within a purchasing interface that is at least partially blocked to a user of the point of sale terminal attempting to purchase the object. The method of claim 1.

7. Implemented on an electronic billboard, wherein the one or more graphical elements comprise information identifying an advertised product and / or a seller of the product. The method of claim 1.

8. implemented in a vehicle, the view being partially blocked to a driver of the vehicle; The method of claim 1.

9. 11. The method of claim 1, wherein the display is implemented with a smart mirror, the obstructed portion of the display further comprising an image of a person standing in front of the smart mirror. The method of claim 1.

10. detecting that a view of the display is no longer blocked by the object; and in response thereto, identifying a previously occluded portion of the display that was previously blocked by the object; relocating at least one of the graphical elements into the previously occluded portion; The method of claim 1 further comprising:

11. 1. A computing device comprising: a processing circuit and a memory comprising instructions executable by the processing circuit, whereby the computing device: detecting that a view of the display is partially blocked by an object; and in response thereto, identifying an occluded portion of the display that is blocked by the object and that comprises at least a portion of one or more graphical elements; identifying an unobstructed portion of the display that is not blocked by the object; and disposing the one or more graphical elements from the occluded portion to the unoccluded portion; 23. A computing device configured to:

12. The computing device of claim 11 , wherein detecting that the view is partially blocked comprises receiving an object detection indication from a weight sensor in front of the display.

13. The computing device of claim 11 , wherein identifying the occluded and unoccluded portions comprises identifying a shape of the object from an image captured by an image sensor.

14. 12. The computing device of claim 11, wherein determining that the occluded portion comprises the one or more graphical elements comprises comparing a shape and position of the object to a shape and position of each of the graphical elements.

15. The computing device of claim 11 , further comprising positioning the one or more graphical elements upon detecting movement of the object into the occluded portion.

16. the computing device comprises a point of sale terminal; the one or more graphical elements are included within a purchasing interface that is at least partially blocked to a user of the point of sale terminal attempting to purchase the object. The computing device of claim 11.

17. the computing device comprises an electronic billboard; the one or more graphical elements comprise information identifying the product being advertised and / or the seller of the product; The computing device of claim 11.

18. The computing device is included within a vehicle; the view is partially blocked to a driver of the vehicle; The computing device of claim 11.

19. detecting that a view of the display is no longer blocked by the object; and in response thereto, identifying a previously occluded portion of the display that was previously blocked by the object; relocating at least one of the graphical elements into the previously occluded portion; The computing device of claim 11 further comprising:

20. A non-transitory computer-readable medium having stored thereon a computer program, the computer program comprising software instructions that, when executed on a computing device, cause the computing device to: detecting that a view of the display is partially blocked by an object; and in response thereto, identifying an occluded portion of the display that is blocked by the object and that comprises at least a portion of one or more graphical elements; identifying an unobstructed portion of the display that is not blocked by the object; and disposing the one or more graphical elements from the occluded portion to the unoccluded portion; A non-transitory computer-readable medium for causing

Images