Providing a continuous virtual space for multiple display devices

Abstract

Disclosed herein are system, method, and computer program product embodiments for rendering a continuous virtual space for multiple display devices. Embodiments operate by identifying projections of virtual cameras onto a virtual plane and determining a display plane area based on the intersection of the projections of the virtual cameras onto the virtual plane. Thereafter, positions of multiple virtual windows associated with multiple real-world display devices on the display plane area are calculated based on (i) characteristics of the display devices and (ii) focal lengths of the virtual cameras. After the virtual windows are placed at their respective display plane area positions, content behind the virtual windows is identified and provided to the corresponding real-world display devices.

Description

Background Art

[0001] Videos are traditionally displayed on multiple real-world display devices in large forums. In doing so, systems are used to illustrate the movement of content in a video across real-world display devices. However, existing systems do not manage well in providing the content of a video to real-world display devices in a coherent manner. For example, existing systems rely on specific hardware and software requirements for storing, generating, and rendering the necessary content provided in the video. As such, existing systems cannot easily adapt to different numbers and / or configurations of display devices. Moreover, existing systems face difficulties in providing the movement of objects in a video across real-world display devices without noticeable delays therebetween. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The accompanying drawings are incorporated herein and constitute a part of the specification.

[0003] Figure 1 A block diagram of a system for providing a continuous virtual space for multiple display devices is illustrated in accordance with some embodiments.

[0004] Figure 2 Multiple display devices displaying a continuous virtual space are illustrated in accordance with some embodiments.

[0005] Figure 3A , 3B 4A and 4B illustrate perspective views of a virtual camera projected into a virtual space according to some embodiments.

[0006] Figure 5 and 6 A flow diagram illustrating a process for providing a continuous virtual space in accordance with some embodiments is illustrated.

[0007] Figure 7 An example computer system for implementing various embodiments is illustrated.

[0008] In the drawings, like reference numbers generally refer to identical or similar elements. Also, generally, the leftmost digit(s) of a reference number identifies the drawing in which the reference number first appears. DETAILED DESCRIPTION

[0009] Provided herein are system, apparatus, device, method, and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for providing a continuous virtual space for multiple display devices.

[0010] The present disclosure describes a system for presenting video across multiple real-world display devices. In the past, such systems would determine, generate, and transmit unique files to real-world display devices for them to present their portions of the video. However, previous underlying processes relied on the storage and computing capabilities of the system and the real-world display devices. Specifically, for example, the system is required to have sufficient storage and computing capabilities to process each unique file, and the real-world display devices are required to play back received files. Moreover, existing systems are unable to provide automation regarding the placement of display devices in the real world, and process virtual scenes based on such placement, so that content is presented across real-world display devices in a coherent manner.

[0011] In some embodiments, when presenting a video, the current system provides virtual windows on a virtual plane in the virtual environment based on their corresponding real-world display devices (e.g., number and configuration). In doing so, the system provides a virtual camera in the virtual environment at a position and location that allows the virtual camera to focus on each virtual window and identify the objects presented therein (e.g., content in the video). As such, the current system is able to identify data for the virtual windows and send it to their corresponding real-world display devices.

[0012] By operating in this manner, unlike existing systems, the embodiments described herein are able to provide appropriate data to real-world display devices to present content in a virtual environment from a shared camera perspective. In doing so, the current system is able to present content across real-world display devices in a coherent manner.

[0013] For example, in some embodiments, the current system can relay a video of a real-world concert to multiple real-world display devices. In doing so, the system can create a virtual window in the virtual world that represents the real-world display device and define a virtual plane area around the virtual window. The system presents the scene of the video within the virtual plane area. Furthermore, the system determines the spatial position of the object relative to the virtual window and transmits only the data required for the specific virtual window to the associated real-world display device. By operating in this manner, the system is able to present objects moving in the scene in a coherent manner that more closely resembles the real world. Moreover, the system is able to send only the data that a specific real-world display device needs to display.

[0014] Figure 1A block diagram of a system 100 for providing a continuous virtual space for multiple display devices is illustrated. The system 100 includes a central server 102, one or more local servers 104, and one or more real-world display devices 106, 108, and 110. The central server 102 communicates with the local servers 104, and the local servers 104 communicate with their corresponding display devices 106. These servers and / or devices can be managed and / or provided by the same entity or different entities. For example, the central server 102 can be managed and / or provided by a third-party entity, and the local servers 104 and display devices 106 can be managed and / or provided by an organization and / or company.

[0015] The central server 102 communicates with the local server 104 via a communication line or medium 112. In turn, the local server 104 communicates with the display device 108 via a communication line or medium 114. The communication lines or mediums 112 and 114 may be private or public.

[0016] As such, the central server 102 stores and / or provides video files for playback by the display devices 106, 108, and 110. As such, the central server 102 can receive video files from external sources. And the central server 102 can act as a distributed playback system when providing the video files to the display devices 106, 108, and 110. The video files can contain a replicated real-world environment. Examples of video files containing a replica of a real-world event include a live recording of a real-world event (e.g., a live concert) or a previously recorded real-world event (e.g., the television show "Numb3rs").

[0017] The central server 102 then identifies content to be presented on the display devices 106, 108, and 110. The content may include a scene (e.g., a series of continuous actions), which may then include one or more objects. Thus, the scene and / or object may appear in real life or in an extended reality (e.g., augmented reality, mixed reality, and / or virtual reality). In this manner, the scene and / or object of the video file is two-dimensional.

[0018] As discussed above, the central server 102 communicates with the local servers 104, which communicate with the sets of display devices 106, 108, and 110. For example, the local server 104a can communicate with a first set of display devices 106, the local server 104b can communicate with a second set of display devices 108, and the local server 104c can communicate with a third set of display devices 110. As such, the local servers 104 and the sets of display devices 106, 108, and 110 can be provided at different locations (e.g., in a building). In turn, each set of display devices 106, 108, and 110 presents the same video file. And each corresponding display device is synchronized with each other (e.g., display devices 106a, 108a, and 110a).

[0019] Thus, depending on the number of sets of display devices 106, 108, and 110, central server 102 and / or local server 104 perform appropriate processing to continuously and synchronously present scenes from a video file. As will be appreciated by one of ordinary skill in the art, a scene represents a crop of a video file.

[0020] For example, if there is only a single set of display devices (e.g., display device 106), then the central server 102 or the local server 104a can perform the processing, or they can act as a single server when doing so. However, if there are multiple sets of display devices 106, 108, and 110 (e.g., display devices 106 and 108), then the central server 102 sends a copy of the data from the original video file to the local servers 104 (e.g., local servers 104a and 104b). In this way, the time data is continuously streamed. The local server 104 thereafter performs processing together with the timing data so that the corresponding display devices from different sets (e.g., display devices 106a and 108a) synchronously present the video file of the central server 102.

[0021] For simplicity, the following disclosure will discuss the central server 102 performing the processing steps. However, in view of the above, it will be readily understood by those skilled in the art that, depending on the number of sets of display devices 106, 108, and 110, the central server 102 and / or the local server 104 can perform the processing steps.

[0022] The central server 102 receives data related to the characteristics of each set of display devices 106, 108, and 110 itself or the relationship between them. The data is pre-stored and can be manually provided by authorized individuals. Example data related to each of the display devices 106, 108, and 110 includes a geographic location on a defined real-world plane (e.g., a specific location on a wall), a display resolution (e.g., 720p, 1080p, 2K, etc.), a physical size (e.g., 32 inches, 50 inches, 60 inches, etc.), a viewing distance (e.g., 6.3-10.4 feet, 6.9-11.5 feet, 7.5-12.5 feet, 8.1-13.5 feet, etc.), a technology type (e.g., a liquid crystal display (LCD), an organic light-emitting diode display (OLED), a plasma display panel (PDP), etc.) and a display type (e.g., two-dimensional). Example data related to the relationship between the display devices 106, 108, and 110 in each set includes the distance between each of the display devices 106, 108, and 110 in each set. For example, the distance between adjacent display devices 106a, 106b, and 106c.

[0023] Based on the distances between adjacent display devices 106 , 108 , and 110 and the sizes of the display devices 106 , 108 , and 110 , the central server 102 determines the configuration (eg, the number of rows and columns) of the display devices 106 , 108 , and 110 . Figure 2 An example spatial configuration of three display devices 202 in a single row is illustrated that provides a continuous virtual space for a viewer 204. However, other configurations are possible as will be apparent to one skilled in the art.

[0024] Return to reference Figure 1 In order to provide a continuous virtual space, the central server 102 identifies a portion (e.g., a video clip) of a video file to be presented to the display devices 106, 108, and 110. As mentioned above, the video file may be a real world environment or a virtual environment. Thus, the central server 102 places a virtual camera at a location and / or position in a replicated real world environment or in a virtual environment.

[0025] Reference now Figure 3A and 3B , the central server 102 ( Figure 1) can then provide a virtual plane 302 in a virtual environment of a portion of the video clip. In some embodiments, multiple virtual planes 302 can be provided in the virtual environment. The virtual planes 302 are physically separated from each other. As such, the virtual planes can be on different axes. In this way, one virtual plane can extend parallel to the y-axis, while another virtual plane can extend parallel to the x-axis. As will be discussed in more detail below, the virtual planes 302 are associated with their own virtual cameras 306 and virtual plane areas 310. In addition, each virtual plane 302 can correspond to a set of real-world display devices 106, 108, and 110. For example, one virtual plane can correspond to a first set of real-world display devices 106, while a second virtual plane can correspond to a second set of real-world display devices 108.

[0026] Figure 3A A perspective front view of a virtual plane 302 in a replicated environment 300 is illustrated. Figure 3B A perspective back view of a virtual plane 302 in a replicated environment 300 is illustrated. The replicated environment 300 includes an object 304. As such, the virtual plane 302 is positioned at least partially in front of the object 304. As the scene of the video may be two-dimensional or three-dimensional, the replicated environment 300 and / or the object 304 may also be two-dimensional or three-dimensional, and the object may be a real world or virtual object.

[0027] Central server 102 ( Figure 1 ) then identifies a virtual plane 302 and places a virtual camera 306 in the replicated environment 300. The virtual camera 306 provides a projection 308 onto the virtual plane 302 that captures at least a portion of the object 304. As such, the projection 308 of the virtual camera 306 intersects the virtual plane 302. In doing so, the intersection defines a display plane area 310 on the virtual plane 302. Following such an approach, the projection 308 of the virtual camera 306 can be any closed shape and define the shape of the display plane area 310 on the virtual plane 302. For example, as shown, the shape of the projection 308 can be a rectangular frustum, and the shape of the display plane area 310 can be a rectangle. Regardless of the shape of the real-world display, the shape of the display plane area 310 can be a rectangle. For example, if the real-world display is a square (i.e., the sides have equal lengths), then the display plane area 310 is still a rectangle.

[0028] The display plane area 310 may be anywhere on the virtual plane 302. The display plane area 310 may be completely in front of the object 304. As will be described in more detail below, the size of the display plane area 310 is based on the projection 308 of the virtual camera 306, the number of virtual windows on the virtual plane 302, and / or the configuration of the virtual windows in the virtual plane 302.

[0029] As such, the projection 308 of the virtual camera 306 onto the virtual plane 302 defines the display plane area 310. And the display plane area 310 defines where the virtual window must be located to provide various fields of view based on the size of the display plane area 310 and the distance of the virtual camera 306 from the virtual plane 302. For example, the field of view may be equal to arctan(0.5×(width of the display plane area 310 / distance of the virtual camera 306 from the display plane area 310))×2. Now referring to Figure 4A and 4B , illustrates a plurality of virtual windows 412 on a virtual plane 402 in a replicated environment 300 (eg, a replicated real world or virtual environment) of a portion of a video clip. Figure 4A A perspective front view of a virtual plane 402 in a replicated environment 400 is illustrated. Figure 4B 400 in the replicated environment 400. As described above, the virtual window 412 is associated with each set of real-world display devices 106, 108, and 110 ( Figure 1 For example, virtual window 412a may correspond to display device 106a, 108a, and / or 110a, virtual window 412b may correspond to display device 106b, 108b, and / or 110b, and virtual window 412c may correspond to display device 106c, 108c, and / or 110c.

[0030] Virtual window 412 has a two-dimensional spatial position (eg, x and y position) within virtual plane 402. In some embodiments, the spatial position of virtual window 412 within virtual plane 402 is based on ( Figure 1) the spatial positions of the real world display devices 106, 108, and 110 in the real world and the characteristics of the real world display devices 106, 108, and 110. This allows the real world display to be reproduced in the virtual world. For example, if the real world display devices 106, 108, and 110 have a specific spatial position in the real world, then the spatial placement of the virtual window 412 in the virtual world is based on this. In other embodiments, the spatial position of the real world display devices 106, 108, and 110 is based on the spatial position of the virtual window 412. For example, the virtual window 412 has a specific spatial position in the virtual world, and the spatial position of the real world display devices 106, 108, and 110 in the real world is based on this. As such, in any of these embodiments, the virtual world has units that are similar to real world units. For example, 1 virtual world unit can be similar to a specific number of inches in the real world (e.g., 1 inch, 3 inches, 6 inches, or 12 inches).

[0031] By operating in this manner, the virtual camera 406 can represent an imaginary camera in the real world that has the same spatial relationship to the real world display devices 106, 108, and 110 as the virtual camera 406 has with respect to the virtual window 412 in the virtual environment.

[0032] As described above, the virtual camera 406 provides a projection onto one of the virtual planes 402, and the projection determines the display plane area 410. In this way, the virtual windows 412 are placed within at least a portion of the display plane area 410. Thus, the virtual camera 406 is able to focus its projection onto each virtual window 412. A When an object 304 is provided in the virtual environment 400 and behind the virtual window 412, the virtual camera 406 captures a view of the virtual window 412 including the object and then provides it to the corresponding display devices 106, 108 and 110 ( Figure 1 ).

[0033] In addition, as described above, the size of the display plane area 410 depends on the projection of the virtual camera 406, the number of virtual windows 412 on the virtual plane 402, and / or the configuration of the virtual windows 412 on the virtual plane 402. As such, the field of view of the virtual camera 406 provided by the projection and the size of the display plane area 410 depend on the size and configuration of the virtual windows 412. For example, the display plane area 410 may have a size that is at least equal to or greater than the configuration of the virtual windows 412 (e.g., 1 (row)×3 (column)). As such, the virtual windows 412 are calibrated based on the size of the display plane area 410, the size of each of the plurality of virtual windows 412, and / or the focal length of the virtual camera 406.

[0034] In addition, after determining the position of the virtual camera 406 relative to the display plane area 410 and / or the virtual window, multiple sub-virtual cameras can be provided for the virtual camera 406. As such, the virtual camera 406 can be considered as a master virtual camera. Thus, the virtual camera 406 (or master virtual camera) can determine the capabilities of the sub-virtual cameras (e.g., viewing distance, focus, clarity, etc.). As such, the sub-cameras can inherit the characteristics of the virtual camera 406.

[0035] According to this method, each child virtual camera can correspond to one of the virtual windows 412. Therefore, since the virtual window 412 is connected to the real world display device 106, 108, 110 ( Figure 1 ), so the child virtual camera can also correspond to the real world display device 106, 108, 110. And each child virtual camera can correspond to a specific real world display device. For example, the child virtual camera can have a 1:1 ratio with the real world display device. In this way, the child virtual camera can provide data related to the content in the virtual window 412 corresponding to it.

[0036] Figure 5 and 6 Flowcharts of methods 500 and 600 for providing a continuous virtual space for multiple display devices according to an embodiment. Methods 500 and 600 may be performed by processing logic, which may include hardware (e.g., circuit systems, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executed on a processing device), or a combination thereof. It should be appreciated that not all steps may be required to perform the disclosure provided herein. In addition, as will be appreciated by those of ordinary skill in the art, some of the steps may be performed simultaneously, or in different Figure 5 and 6 Execute in the order shown in .

[0037] Reference now Figure 5 , will refer to Figure 1 and 2 The method 500 is described for providing a continuous virtual space to only a single set of display devices 106, 108, and 110 (eg, display device 106). However, the method 500 is not limited to that example embodiment.

[0038] At 502, the central server 102 obtains display information associated with one or more sets of real-world display devices 106, 108, and 110. As described above, each set of real-world display devices 106, 108, and 110 is provided to display the same video file. Figure 2As shown in , a collection of display devices provides a continuous virtual space for a viewer 204. As such, the central server 102 receives the position (e.g., in centimeters), rotation (e.g., in degrees), size (e.g., in centimeters), resolution (e.g., in pixels), and aspect ratio (e.g., in centimeters) of each display device in the collection and the collection itself.

[0039] At 504, the central server 102 determines whether there is a single set of real-world display devices 106, 108, and 110 (e.g., only display device 106) or multiple sets of display devices 106, 108, and 110 (e.g., display devices 106 and 108). If the central server 102 determines that there are multiple sets of display devices 106, 108, and 110, then the method 500 proceeds to 506. However, if the central server 102 determines that there is only a single set of display devices 106, 108, and 110, then the method 500 proceeds to 508.

[0040] At 506, the central server 102 forwards the content to be viewed on the display real-world display devices 106, 108, and 110 along with the timing data to one or more local servers 104. To this end, the central server 102 sends the timestamp and / or time code of the local servers 104. This will allow corresponding display devices (e.g., display devices 106a and 108a) of each of the multiple sets of display devices 106, 108, and 110 to present the same or similar data simultaneously.

[0041] At 508 , the central server 102 or the local server 104 determines one or more characteristics of a replicated environment to be shared between each of the set of real-world display devices 106 , 108 , and 110 based on the content to be viewed on the display devices 106 , 108 , and 110 .

[0042] Thus, in some embodiments where there is a single set of display devices 106, 108, or 110 (e.g., display device 106), the central server 102 or the corresponding local server 104 determines the characteristics of the particular set of real-world display devices 106, 108, or 110. In other embodiments where there are multiple sets of real-world display devices 106, 108, and 110 (e.g., display devices 106 and 108), the local server 104 determines the characteristics of their corresponding real-world display devices 106, 108, and 110.

[0043] In 510, the central server 102 or the local server 104 creates a main virtual camera from the virtual window provided on the virtual plane and corresponds to each set of real-world display devices. As discussed above, a main virtual camera can be provided for the virtual plane. And after determining the characteristics of the main virtual camera, multiple sub-virtual cameras corresponding to the virtual windows can be provided.

[0044] In 512 , the central server 102 or the local server 104 modifies the viewport of the main camera and / or the viewports of its child cameras based on characteristics associated with the plurality of windows in the virtual environment.

[0045] In 514 , the central server 102 or the local server 104 receives data from the main camera or the virtual camera based on the main camera or the virtual camera's viewing of the content in the virtual window 412 .

[0046] In 516 , the central server 102 or the local server 104 provides data to the plurality of real-world display devices 106 , 108 , and 110 corresponding to the respective plurality of virtual windows 412 .

[0047] As explained above, if there is a single set of display devices 106, 108, and 110 (e.g., display device 106), then the central server 102 or the appropriate local server (e.g., local server 104a) performs steps 508, 510, and 512. In this manner, as described above, the central server 102 and the appropriate local server 104 can be combined into a single server. However, if there are multiple sets of display devices 106, 108, and 110 (e.g., display devices 106 and 108), then the local server 104 receives timing data from the central server 102 and performs steps 508, 510, 512, and 514 based on the timing data.

[0048] Reference now Figure 6 , method 600 will refer to Figure 1 , 3, 4, and 5. However, method 600 is not limited to those example embodiments.

[0049] At 602 , the central server 102 or the local server 104 identifies a projection 308 of a virtual camera 306 onto a virtual plane 302 provided in a replicated environment 300 (eg, a replicated real world or virtual environment).

[0050] In 604 , the central server 102 or the local server 104 determines the display plane area 310 based on the intersection of the projections 308 of the virtual camera 306 onto the virtual plane 302 .

[0051] In 606, the central server 102 or the local server 104 calculates the position of each of the plurality of virtual windows 412 provided on the display plane area 410 based on (i) one or more characteristics of the plurality of real-world display devices 106, 108, and 110 and (ii) the focal length of the virtual camera 406. Thus, each of the plurality of virtual windows 412 relates to / corresponds to a respective one of the plurality of display devices 106, 108, and 110.

[0052] In 608 , the central server 102 or the local server 104 places the plurality of virtual windows 412 at the locations on the display plane area 410 .

[0053] At 610, the central server 102 or the local server 104 identifies content that is at least partially behind at least one of the plurality of virtual windows 412. The content may be an object 304, which may be two-dimensional or three-dimensional.

[0054] In 612 , the central server 102 or the local server 104 provides data related to at least one virtual window of the plurality of virtual windows 412 having the content to a corresponding one of the plurality of display devices 106 , 108 , and 110 .

[0055] As mentioned above Figure 6 As described, steps 602, 604, 606, 608, 610, and 612 may be performed by the central server 102 and / or the local server 104. For example, in some embodiments, steps 602, 604, 606, 608, 610, and / or 612 are performed by the central server 102 and provided to the local server 104.

[0056] For example, various embodiments may be implemented using one or more well-known computer systems, such as Figure 7 . For example, one or more computer systems 700 may be used to implement any of the embodiments discussed herein, and combinations and sub-combinations thereof.

[0057] Computer system 700 may include one or more processors (also referred to as central processing units or CPUs), such as processor 704. Processor 704 may be connected to a communications infrastructure or bus 706.

[0058] The computer system 700 may also include (one or more) user input / output devices 703, such as a monitor, keyboard, pointing device, etc., which may communicate with the communication infrastructure 706 via (one or more) user input / output interfaces 702.

[0059] One or more of the processors 704 may be a graphics processing unit (GPU). In an embodiment, a GPU may be a processor that is a dedicated electronic circuit designed to process math-intensive applications. A GPU may have a parallel structure that is efficient for processing large blocks of data (such as math-intensive data common to computer graphics applications), images, videos, etc. in parallel.

[0060] The computer system 700 may also include a main memory 708 or primary memory, such as a random access memory (RAM). The main memory 708 may include one or more levels of cache. The main memory 708 may store control logic (ie, computer software) and / or data therein.

[0061] The computer system 700 may also include one or more secondary storage devices or memories 710. The secondary storage 710 may include, for example, a hard disk drive 712 and / or a removable storage device or drive 714. The removable storage drive 714 may be a floppy disk drive, a magnetic tape drive, an optical disk drive, an optical storage device, a tape backup device, and / or any other storage device / drive.

[0062] The removable storage drive 714 can interact with a removable storage unit 718. The removable storage unit 718 may include a computer usable or readable storage device having computer software (control logic) and / or data stored thereon. The removable storage unit 718 may be a floppy disk, a magnetic tape, an optical disk, a DVD, an optical storage disk, and / or any other computer data storage device. The removable storage drive 714 can read from and / or write to the removable storage unit 718.

[0063] Secondary memory 710 may include other components, devices, assemblies, tools, or other methods for allowing computer programs and / or other instructions and / or data to be accessed by computer system 700. Such components, devices, assemblies, tools, or other methods may include, for example, a removable storage unit 722 and an interface 720. Examples of removable storage unit 722 and interface 720 may include a program cartridge and cartridge interface (such as those found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and / or any other removable storage unit and associated interface.

[0064] The computer system 700 may also include a communication or network interface 724. The communication interface 724 may enable the computer system 700 to communicate and interact with any combination of external devices, external networks, external entities, etc. (individually and collectively referenced by reference numeral 728). For example, the communication interface 724 may allow the computer system 700 to communicate with an external or remote device 728 via a communication path 726, which may be wired and / or wireless (or a combination thereof) and may include any combination of a LAN, a WAN, the Internet, etc. Control logic and / or data may be transmitted to and from the computer system 700 via the communication path 726.

[0065] The computer system 700 may also be any of a personal digital assistant (PDA), a desktop workstation, a laptop or notebook computer, a netbook, a tablet computer, a smart phone, a smart watch or other wearable device, an appliance, part of the Internet of Things, and / or an embedded system, just to name a few non-limiting examples, or any combination thereof.

[0066] Computer system 700 can be a client or server, accessing or hosting any applications and / or data through any delivery paradigm, including but not limited to remote or distributed cloud computing solutions; local or on-premises software ("on-premises" cloud-based solutions); "as a service" models (e.g., content as a service (CaaS), digital content as a service (DCaaS), software as a service (SaaS), hosted software as a service (MSaaS), platform as a service (PaaS), desktop as a service (DaaS), framework as a service (FaaS), backend as a service (BaaS), mobile backend as a service (MBaaS), infrastructure as a service (IaaS), etc.); and / or a hybrid model including any combination of the above examples or other services or delivery paradigms.

[0067] Any applicable data structures, file formats, and schemas in the computer system 700 may be derived from standards including, but not limited to, JavaScript Object Notation (JSON), Extensible Markup Language (XML), another markup language (YAML), Extensible Hypertext Markup Language (XHTML), Wireless Markup Language (WML), MessagePack, XML User Interface Language (XUL), or any other functionally similar representations, alone or in combination. Alternatively, proprietary data structures, formats, or schemas may be used either alone or in combination with known or open standards.

[0068] In some embodiments, a tangible, non-transitory device or article of manufacture including a tangible, non-transitory computer-usable or readable medium having control logic (software) stored thereon may also be referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system 700, main memory 708, auxiliary memory 710, and removable storage units 718 and 722, as well as tangible articles of manufacture implementing any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as computer system 700), may cause such data processing devices to operate as described herein.

[0069] Based on the teachings contained in this disclosure, it will be clear to one skilled in the relevant art(s) how to use the Figure 7 Embodiments of the present disclosure may be made and used with data processing devices, computer systems, and / or computer architectures other than those shown in . In particular, the embodiments may operate using software, hardware, and / or operating system implementations different from those described herein.

[0070] It should be appreciated that the detailed description section, rather than any other section, is intended to be used to interpret the claims. The other sections may set forth one or more but not all exemplary embodiments contemplated by the inventor(s), and therefore, are not intended to limit the present disclosure or the appended claims in any way.

[0071] Although the present disclosure describes exemplary embodiments for exemplary fields and applications, it should be understood that the present disclosure is not limited thereto. Other embodiments and modifications thereto are possible and within the scope and spirit of the present disclosure. For example, but not limiting the generality of this paragraph, the embodiments are not limited to the software, hardware, firmware, and / or entities shown in the figures and / or described herein. In addition, the embodiments (whether or not explicitly described herein) have significant utility for fields and applications beyond the examples described herein.

[0072] Embodiments have been described herein with the aid of functional building blocks that illustrate implementations of specified functions and relationships thereof. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries may be defined so long as the specified functions and relationships (or their equivalents) are appropriately performed. Moreover, alternative embodiments may perform functional blocks, steps, operations, methods, etc., in an order different from that described herein.

[0073] References herein to "one embodiment", "embodiment", "example embodiment" or similar phrases indicate that the described embodiments may include specific features, structures or characteristics, but not necessarily each embodiment includes the specific features, structures or characteristics. Moreover, such phrases do not necessarily refer to the same embodiment. In addition, when describing specific features, structures or characteristics in conjunction with an embodiment, it will be within the knowledge of (one or more) relevant art technicians to combine such features, structures or characteristics into other embodiments, whether or not explicitly mentioned or described herein. In addition, the expressions "coupled" and "connected" together with their derivatives can be used to describe some embodiments. These terms do not have to be synonymous with each other. For example, the terms "connected" and / or "coupled" can be used to describe some embodiments to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" can also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0074] The breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

1. A computer-implemented method for providing a continuous environment for multiple display devices, the computer-implemented method include: identifying, by at least one processor, a projection of a virtual camera onto a virtual plane, wherein the virtual plane is provided in a replicated real world environment or in a virtual environment; determining, by the at least one processor, a display plane area based on an intersection of projections of the virtual cameras onto the virtual plane; calculating, by the at least one processor, a position of each of a plurality of virtual windows on the display plane area based on (i) one or more characteristics of a plurality of real-world display devices and (ii) a focal length of a virtual camera, wherein at least one of the plurality of virtual windows corresponds to a corresponding real-world display device of the plurality of real-world display devices, and wherein the position of the virtual camera in the replicated real-world environment is based on the one or more characteristics of the plurality of real-world display devices in the replicated real-world environment; placing, by the at least one processor, the plurality of virtual windows at the locations on the display plane area; identifying, by the at least one processor, content located at least partially behind at least one virtual window of the plurality of virtual windows; as well as providing, by the at least one processor, data related to the at least one virtual window having the content among the plurality of virtual windows to a corresponding real-world display device among the plurality of real-world display devices, Wherein at least one of identifying, determining, calculating, placing and providing is performed by one or more computers.

2. The computer-implemented method of claim 1, wherein the content is captured by the virtual camera.

3. The computer-implemented method of claim 1, wherein the content comprises a three-dimensional virtual object.

4. The computer-implemented method of claim 1, further comprising: include: determining, by the at least one processor, a position of a virtual camera within the replicated real-world environment; as well as The virtual camera is rendered, by the at least one processor, in the replicated real world environment at the location of the virtual camera in the replicated real world environment.

5. The computer-implemented method of claim 4, wherein the position of the virtual camera in the replicated real-world environment is based on a focal length of the virtual camera to an area of ​​a display plane.

6. The computer-implemented method of claim 1, wherein the projection of the virtual camera is a defined shape.

7. The computer-implemented method of claim 6, wherein the defined shape is a rectangular frustum of a pyramid.

8. The computer-implemented method of claim 1, wherein the display plane area encompasses at least each virtual window of the plurality of virtual windows.

9. The computer-implemented method of claim 1, wherein each of the plurality of virtual windows corresponds to a respective real-world display device of the plurality of real-world display devices.

10. The computer-implemented method of claim 9, wherein the plurality of real-world display devices comprises a first set of real-world display devices and a second set of real-world display devices, the first set and the second set each providing a continuous environment.

11. The computer-implemented method of claim 10, wherein the virtual camera comprises a plurality of virtual cameras, and Each of the plurality of virtual cameras is focused on a corresponding virtual window among the plurality of virtual windows.

12. The computer-implemented method of claim 11, wherein the first set of real-world display devices, the second set of real-world display devices, and the plurality of virtual windows have a same configuration.

13. The computer-implemented method of claim 10, wherein each virtual window of the plurality of virtual windows corresponds to a respective real-world display device in each of the first set of real-world display devices and the second set of real-world display devices.

14. The computer-implemented method of claim 1, wherein the one or more characteristics of the plurality of real-world display devices include resolution, position, and size of the plurality of real-world display devices.

15. A system for providing a continuous environment for a plurality of display devices, include: Memory; as well as at least one processor coupled to the memory and configured to: identifying a projection of a virtual camera onto a virtual plane, wherein the virtual plane is provided in the replicated environment; determining a display plane area based on an intersection of projections of the virtual camera onto the virtual plane; calculating a position of each of a plurality of virtual windows on the display plane area based on (i) one or more characteristics of a plurality of real-world display devices and (ii) a focal length of a virtual camera, wherein at least one of the plurality of virtual windows corresponds to a corresponding real-world display device of the plurality of real-world display devices, and wherein the position of the virtual camera in the replicated environment is based on the one or more characteristics of the plurality of real-world display devices in the replicated environment; placing the plurality of virtual windows at the positions on the display plane area; identifying content located at least partially behind at least one virtual window of the plurality of virtual windows; as well as Data related to the at least one virtual window having the content among the plurality of virtual windows is provided to a corresponding real-world display device among the plurality of real-world display devices.

16. The system of claim 15, wherein the at least one processor is further configured to: Determining a position of the virtual camera in the replicated environment; and rendering the virtual camera in the replicated environment at the position of the virtual camera in the replicated environment.

17. The system of claim 15, wherein the projection of the virtual camera is a rectangular frustum.

18. The system of claim 15, wherein the one or more characteristics of the plurality of real-world display devices include resolution, position, and size of the plurality of real-world display devices.

19. A non-transitory computer-readable device having instructions stored thereon, the instructions, when executed by at least one computing device, causing the at least one computing device to perform operations comprising: identifying a projection of a virtual camera onto a virtual plane, wherein the virtual plane is provided in a replicated real world environment or in a virtual environment; determining a display plane area based on an intersection of projections of the virtual camera onto the virtual plane; calculating a position of each of a plurality of virtual windows on the display plane area based on (i) one or more characteristics of a plurality of real-world display devices and (ii) a focal length of a virtual camera, wherein at least one of the plurality of virtual windows corresponds to a corresponding real-world display device of the plurality of real-world display devices, and wherein the position of the virtual camera in the replicated real-world environment is based on the one or more characteristics of the plurality of real-world display devices in the replicated real-world environment; placing the plurality of virtual windows at the positions on the display plane area; identifying content located at least partially behind at least one virtual window of the plurality of virtual windows; as well as Data related to the at least one virtual window having the content among the plurality of virtual windows is provided to a corresponding real-world display device among the plurality of real-world display devices.

Images