Remote pointer for simulator

Abstract

A computer-implemented method for remotely pointing to an area of ​​interest within a simulated environment is provided, the process including the steps of receiving coordinates of at least one point selected within a given image of at least a portion of the simulated environment, the given image being displayed on a display unit; identifying the area of ​​interest within the simulated environment based on the received coordinates of the at least one point; and providing control signals to a projector to visually distinguish the area of ​​interest within the simulated environment.

Description

Technical Field

[0005] ,

[0001] The present invention relates to the field of simulation, and more particularly to remote access and / or control of a simulation environment.

Background Art

[0002] Aircraft simulators are used to train and evaluate pilots and copilots. For this purpose, the person responsible for training and evaluating the pilot or copilot is physically present within the simulator.

[0003] However, it may be desirable to provide remote visual access to the simulation environment to third parties such as existing clients, potential clients, regulatory agencies, and flight instructors. This is particularly true during a pandemic. There are several custom solutions that provide such access. However, their design and implementation are time-consuming and / or expensive.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Therefore, there is a need for an improved method and system for remotely pointing to areas of interest within a simulation environment.

Means for Solving the Problems

[0005] A first broad embodiment provides a computer-operated method for remotely pointing to a region of interest within a simulation environment, the process comprising: receiving the coordinates of at least one selected point in a given image of at least a portion of the simulation environment, the given image being displayed on a display unit; identifying a region of interest within the simulation environment based on the received coordinates of the at least one point; and providing a control signal to a projector to visually distinguish the region of interest within the simulation environment.

[0006] In one embodiment, the step of identifying a region of interest in the simulator includes the step of mapping received coordinates to a pointer area in a coordinate system associated with the projector, and the step of providing a control signal includes the step of generating an overlay image based on the pointer area, wherein the overlay image encompasses the region of interest, and the step of providing a control signal to the projector to project the overlay image on the simulator.

[0007] In one embodiment, the step of mapping received coordinates to a pointer area includes the steps of mapping received coordinates to at least one given point in the coordinate system of the simulation environment, and mapping at least one given point to a pointer area in the coordinate system associated with the projector.

[0008] In one embodiment, the method further includes the steps of establishing a connection between a first video conferencing client running on a first computer and a second video conferencing client running on a second computer over a telecommunications network; transmitting a given image to the second computer through the first video conferencing client, wherein the second computer provides the given image for display on a remote display unit, the step of receiving coordinates includes receiving coordinates from the second computer through the first video conferencing client over a secure public network.

[0009] In one embodiment, the telecommunications network includes one of a public network, a secure public network, a private network, and a secure private network.

[0010] In one embodiment, the step of providing a control signal includes the step of selecting a projector from among a plurality of available projectors based on received coordinates, and the step of providing a control signal to the selected projector.

[0011] In one embodiment, the step of providing a control signal includes the step of determining a target position for a projector, the target position ensuring that the area of ​​interest is included within the projector's field of view, and the step of providing a control signal that further indicates the target position.

[0012] In one embodiment, the method further includes the step of receiving a given image of at least a portion of the simulation environment from a camera located within the simulation environment.

[0013] In one embodiment, the method further includes the step of generating a given image of at least a portion of the simulation environment.

[0014] In one embodiment, the step of generating a given image is performed using a 3D model of the simulation environment.

[0015] In one embodiment, the step of receiving coordinates includes receiving coordinates for a plurality of points selected in a given image; the step of identifying regions of interest includes identifying a plurality of regions of interest based on the received coordinates for the plurality of points; and the step of providing control signals includes providing control signals to a projector so as to visually and simultaneously distinguish a plurality of regions of interest in a simulation environment.

[0016] In one embodiment, the method further includes the step of selecting at least one characteristic of a pointer area, and the control signal exhibits at least one characteristic.

[0017] In one embodiment, at least one characteristic includes at least one of shape, strength, and color.

[0018] In one embodiment, the method further includes the step of providing remote visibility to the simulation environment by establishing a connection between a first video conferencing client running on a first computer and a second video conferencing client running on a second computer over a secure public network, receiving a first live camera feed on the first computer over a secure private network, the live camera feed encompassing a first portion of the simulation environment and captured by the first camera, and the second computer, located outside the secure private network, displaying the first live camera feed and at least one of a simulation control interface, a computer-generated representation of a second portion of the simulation environment, and a second live camera feed captured by the second camera and encompassing a third portion of the simulation environment distinct from the first portion, within a viewer application running on the first computer, and allowing the viewer application remote visual access of the second video conferencing client through the first video conferencing client.

[0019] In one embodiment, the method further includes the steps of: receiving a live audio feed from a microphone located within a simulation environment in a first computer; and transmitting the live audio feed to a second computer via a first video conferencing client over a secure public network, so as to enable the second computer to play the live audio feed in substantially real time.

[0020] In one embodiment, the method further includes the steps of: receiving a live audio feed from a second computer through a first video conferencing client in a first computer; and transmitting the received live audio feed to a speaker installed in a simulation environment.

[0021] In one embodiment, the simulation environment includes a flight simulator, and a first part of the simulation environment encompasses the equipment and control part of the flight simulator.

[0022] In one embodiment, at least one video feed includes at least two video feeds.

[0023] In one embodiment, a given one of the at least two video feeds encompasses an internal part of the flight simulator, and the internal part of the flight simulator includes the screen of the flight simulator on which an image of the computer-generated environment is displayed.

[0024] In one embodiment, the flight simulator includes a motion-based flight simulator, and a given one of the at least two video feeds encompasses an external part of the motion-based flight simulator.

[0025] In one embodiment, the viewer application is further configured to provide a view of the simulation control interface of the simulator.

[0026] In one embodiment, the simulation control interface includes the interface of the instructor operation station, and the second live camera feed encompasses the instructor operation station.

[0027] In one embodiment, the method further includes receiving, from the simulation engine, a view of the simulation control interface.

[0028] In one embodiment, the viewer application is further configured to provide a view of the interface of the data analysis platform.

[0029] In one embodiment, the view of the interface of the data analysis platform corresponds to an image of a display unit on which the interface of the data analysis platform is displayed, and the image of the display is captured by a further camera connected to the private network.

[0030] In one embodiment, the method further includes receiving, from a server hosting the data analysis platform, a view of the interface of the data analysis platform.

[0031] In one embodiment, the method further includes permitting control of the viewer application to a second video conference client.

[0032] In one embodiment, the method further includes permitting control of the operating system interface to a second video conference client.

[0033] In one embodiment, the method further includes permitting control of the interface of the data analysis platform to a second video conference client.

[0034] In one embodiment, the method further includes receiving, from a first video conference client, an instruction to project a visual indication in the environment, and controlling a projector installed in the simulator to project the visual indication.

[0035] In another broader embodiment, a system is provided for remotely pointing to a region of interest within a simulator, the system comprising a processor and a non-temporary storage medium operably connected to the processor, the non-temporary storage medium containing computer-readable instructions, wherein when the processor executes a computer-readable instruction, it receives the coordinates of at least one point selected in a given image of at least a portion of the simulation environment, the given image is displayed on a display unit, and based on the received coordinates of at least one point, it identifies a region of interest within the simulation environment and is configured to provide a control signal to a projector to visually distinguish the region of interest within the simulation environment.

[0036] In one embodiment, the processor is configured to map the received coordinates to a pointer area in a coordinate system associated with the projector, generate an overlay image based on the pointer area, and provide control signals to the projector so that the overlay image encompasses the region of interest and projects the overlay image on the simulator.

[0037] In one embodiment, the processor is configured to map received coordinates to at least one given point in the coordinate system of the simulation environment, and to map at least one given point to a pointer area in the coordinate system associated with the projector.

[0038] In one embodiment, the processor is configured to establish a connection between a first video conferencing client running on a first computer and a second video conferencing client running on a second computer over a telecommunications network, transmit a given image to the second computer through the first video conferencing client, and the second computer is configured to provide the given image for display on a remote display unit, and receiving coordinates includes receiving coordinates from the second computer through the first video conferencing client over a secure public network.

[0039] In one embodiment, the telecommunications network includes one of a public network, a secure public network, a private network, and a secure private network.

[0040] In one embodiment, the processor is further configured to select a projector from among several available projectors based on the received coordinates and to provide a control signal to the selected projector.

[0041] In one embodiment, the processor is further configured to determine a target position for the projector, the target position ensuring that the area of ​​interest is included within the projector's field of view, and to provide a control signal that further indicates the target position.

[0042] In one embodiment, the processor is further configured to receive a given image of at least a portion of the simulation environment from a camera located within the simulation environment.

[0043] In one embodiment, the processor is further configured to generate a given image of at least a portion of the simulation environment.

[0044] In one embodiment, the processor is configured to generate a given image using a 3D model of the simulation environment.

[0045] In one embodiment, the processor is configured to receive coordinates for a plurality of points selected within a given image, identify a plurality of regions of interest based on the received coordinates for the plurality of points, and provide control signals to the projector to visually and simultaneously distinguish the plurality of regions of interest within the simulation environment.

[0046] In one embodiment, the processor is further configured to select at least one characteristic of the pointer area, and the control signal exhibits at least one characteristic.

[0047] In one embodiment, at least one characteristic includes at least one of shape, strength, and color.

[0048] In one embodiment, the system is further adapted to provide remote visibility to the simulation environment, and the processor further establishes a connection between a first video conferencing client running on a first computer and a second video conferencing client running on a second computer over a secure public network, and receives a first live camera feed on the first computer over a secure private network, the live camera feed encompassing a first portion of the simulation environment and captured by the first camera, and the second computer is located outside the secure private network and displays, within a viewer application running on the first computer, the first live camera feed and at least one of the following: a simulation control interface, a computer-generated representation of a second portion of the simulation environment, and a second live camera feed captured by the second camera and encompassing a third portion of the simulation environment distinct from the first portion, and is configured to allow the viewer application visual access of the second video conferencing client through the first video conferencing client.

[0049] In one embodiment, the processor is further configured to receive a live audio feed from a microphone present in the simulation environment in the first computer and to transmit the live audio feed to the second computer via the first video conferencing client over a secure public network, enabling the second computer to play the live audio feed in substantially real time.

[0050] In one embodiment, the processor is further configured in the first computer to receive a live audio feed from the second computer via the first video conferencing client and to transmit the received live audio feed to a speaker installed in the simulation environment.

[0051] In one embodiment, the simulation environment includes a flight simulator, and the first part of the simulation environment comprises the equipment and control parts of the flight simulator.

[0052] In one embodiment, at least one video feed includes at least two video feeds.

[0053] In one embodiment, a given one of at least two video feeds includes an interior portion of a flight simulator, the interior portion of the flight simulator includes a screen of the flight simulator displaying computer-generated images of the environment.

[0054] In one embodiment, the flight simulator includes a motion-based flight simulator, where a given one of at least two video feeds includes an external portion of the motion-based flight simulator.

[0055] In one embodiment, the viewer application is further configured to provide a view of the simulator's simulation control interface.

[0056] In one embodiment, the simulation control interface includes the interface of the instructor operation station, and the second live camera feed includes the instructor operation station.

[0057] In one embodiment, the processor is further configured to receive a view of the simulation control interface from the simulation engine.

[0058] In one embodiment, the viewer application is further configured to provide a view of the interface of the data analysis platform.

[0059] In one embodiment, a view of the data analysis platform interface corresponds to an image of a display unit on which the data analysis platform interface is displayed, and the image of the display is captured by an additional camera connected to a private network.

[0060] In one embodiment, the processor is further configured to receive a view of the data analysis platform interface from a server hosting the data analysis platform.

[0061] In one embodiment, the processor is further configured to allow a second video conferencing client to control the viewer application.

[0062] In one embodiment, the processor is further configured to allow a second video conferencing client to control the operating system interface.

[0063] In one embodiment, the processor is further configured to allow a second video conferencing client to control the interface of the data analysis platform.

[0064] In one embodiment, the processor is further configured to receive a command from a first video conferencing client to project a visual indication within the environment, and to control a projector installed in the simulator to project the visual indication.

[0065] In a more extensive manner, a computer program product is provided for remotely pointing to a region of interest within a simulation environment, the computer program product includes a computer-readable memory storing computer executable instructions, the computer executable instructions, when executed by a processor, perform the steps of: receiving the coordinates of at least one point selected in a given image of at least a portion of the simulation environment, the given image being displayed on a display unit; identifying a region of interest within the simulation environment based on the received coordinates of at least one point; and providing a control signal to a projector to visually distinguish the region of interest within the simulation environment.

[0066] In one embodiment, the computer program product is further adapted to provide remote visibility to the simulation environment, and the computer-readable memory stores further computer-executable instructions, which are executed by the processor. The steps include establishing a connection between a first video conferencing client running on a first computer and a second video conferencing client running on a second computer over a secure public network, and receiving a first live camera feed on the first computer over a secure private network, wherein the live camera feed includes a first part of the simulation environment and a first camera The method steps include: capturing the second computer, which is located outside the secure private network; displaying, within a viewer application running on the first computer, at least one of a first live camera feed, a simulation control interface, a computer-generated representation of a second part of the simulation environment, and a second live camera feed captured by the second camera, which includes a third part of the simulation environment distinct from the first part; and granting the viewer application visual access to the second video conferencing client through the first video conferencing client.

[0067] Further features and advantages of the present invention will become apparent from the following detailed description, which will be adopted in conjunction with the accompanying drawings. [Brief explanation of the drawing]

[0068] [Figure 1] This flowchart illustrates a computer-based method for providing remote visibility into a simulation environment, as demonstrated by the examples. [Figure 2] This diagram shows an exemplary pole installed inside the cockpit, which has multiple cameras mounted on it to capture images of different parts of the cockpit. [Figure 3] This figure shows an exemplary image generated by a viewer application, containing six regions. [Figure 4] Figure 3 is an illustrative diagram showing an exemplary video conferencing client interface with the image displayed. [Figure 5] This is a block diagram illustrating a system that provides remote visibility to a simulation environment, based on an embodiment. [Figure 6] This is a conceptual diagram illustrating an exemplary cockpit of an equipped aircraft simulator according to an embodiment. [Figure 7] This is a conceptual diagram illustrating an interface for simultaneously displaying four different images, based on an embodiment. [Figure 8] This is a conceptual diagram illustrating four interfaces, each of which will be displayed on a different display unit, according to the embodiment. [Figure 9] This flowchart illustrates a method for remotely illuminating a region of interest within a simulator, based on an example. [Figure 10] This flowchart illustrates a method for remotely controlling a projector to illuminate an area of ​​interest within a simulator, according to an embodiment. [Figure 11] This figure shows an example of a computer-generated image illustrating the simulator portion. [Figure 12] This figure shows an exemplary image captured by a camera located within the simulator, with the region of interest illuminated. [Figure 13] This is a block diagram illustrating a system for remotely controlling a projector to illuminate a region of interest within a simulator, according to an embodiment. [Modes for carrying out the invention]

[0069] It should be noted that throughout the attached drawings, similar features are identified by the same reference numerals.

[0070] In one embodiment, the methods and systems described below may be used, for example, in the context of an aircraft simulator for training pilots and / or co-pilots. The aircraft simulator typically includes a training room that mimics an aircraft cockpit. The training room is typically movable to simulate the movement of a simulated aircraft. The training room is equipped with equipment and controls that typically correspond to, or mimic, those found in real equipment and controls within an aircraft. The training room also typically includes two seats, one for the pilot and one for the co-pilot. Furthermore, the training room typically includes an instructor control station located behind the pilot and co-pilot seats, which allows the instructor to control the simulation and / or monitor the pilot and co-pilot's performance during the simulation. For example, the instructor may set the parameters of the simulation or load a given simulation via the instructor control station. The instructor operation station may also be used by agents of regulatory agencies, such as the U.S. Federal Aviation Administration (FAA), to evaluate pilots and / or co-pilots in order to renew their certifications.

[0071] The typical architecture of an aircraft simulator requires an instructor or agent to be physically present in the simulator's training room to direct, guide, and / or evaluate the pilot and / or co-pilot during the simulation. However, in some specific circumstances, such as during a pandemic, it may be advantageous to remotely direct, guide, and / or evaluate the pilot and / or co-pilot during the simulation. To this end, the Method and System provides a third-party human, such as an instructor or agent, with remote visibility or remote visual access to at least a portion of the cockpit or flight deck, including the equipment and controls located within the simulator. The Method and System may further enable the instructor or agent to remotely control the simulation engine. The Method and System may also enable communication between the pilot and / or co-pilot and a third party.

[0072] In one embodiment, the technology is implemented using a pre-built video conferencing platform, which enables a rapid and / or inexpensive implementation regardless of the location and system used by a third party.

[0073] Figure 1 illustrates one embodiment of Method 10 for providing remote visibility to a simulation environment. It should be understood that Method 10 is performed by at least one processor. As described above, the simulation environment includes at least equipment and controls in the placement of a user, such as a pilot, during the simulation. The simulation environment is further provided with at least one camera positioned and oriented to capture at least a portion of the simulator, such as at least a portion of the equipment and controls. The camera is connected to a first computer machine via a private network. The first computer machine is also connected to a second computer machine located outside the private network via a telecommunications network. The second computer machine does not have access to the private network. The first and second computer machines communicate via a video conferencing platform using the telecommunications network.

[0074] In step 12, the first computer machine receives a first video feed from the camera over a private network. The first video feed encompasses at least a first portion of the simulator; that is, the camera captures images of at least a portion of the simulator's interior space, and the captured images are transmitted to the first computer machine. For example, the camera may be positioned behind the pilot and co-pilot seats, and as a result, the video feed may encompass the pilot, co-pilot, and cockpit portions.

[0075] In step 14, the viewer application and the first video conferencing client are running simultaneously on a first computer machine. The first computer machine includes a first display unit, and the viewer application is configured to display the video feed received in step 12 on the first display unit. For example, the viewer application may be configured to generate an interface for displaying at least one video feed thereon and / or to format the video feeds so that they can be displayed on the first display unit connected to the first computer machine. The first video conferencing client is configured to send and receive data with the second video conferencing client over a telecommunications network.

[0076] In one embodiment, the first computer machine further receives, in step 12, a simulation control interface, a computer-generated representation of the second part of the simulator, and / or a second live camera feed that captures a second camera and includes a different part of the simulator from the part of the simulator included in the first video feed. In this case, in step 14, the simulation control interface, the computer-generated representation of the second part of the simulator, and / or the second live camera feed are displayed in the viewer application in addition to the first video feed.

[0077] It should be understood that a viewer application is configured to display images, such as video feeds or any image. In one embodiment, the viewer application has the ability to handle various graphic file formats. In one embodiment, the viewer application is configured to render images according to display characteristics such as color depth, display resolution, and color profile.

[0078] In step 16, the second video conferencing client running on the second computer machine is granted access to the viewer application through the first video conferencing client running on the first computer machine. As a result, the video feed is transmitted from the first video conferencing client to the second video conferencing client via the telecommunications network. Consequently, the same image displayed on the first display unit connected to the first computer machine may be remotely displayed on the second display unit connected to the second computer machine, and a third party may have remote visibility into the simulated environment while using the video conferencing platform.

[0079] In one embodiment, Method 10 further includes the step of establishing a connection between a first video conferencing client running on a first computer and a second video conferencing client running on a second computer over a secure public network.

[0080] In one embodiment, the first and second video conferencing clients are part of a ready-made video conferencing platform configured to at least transmit video feeds between computer machines participating in a video conferencing session. For example, the ready-made video conferencing platform may be Teams®, Webex®, or Google Meet®.

[0081] It should be understood that the camera's position and orientation relative to the simulator are selected according to the desired view of the simulator. As described above, the camera may be located inside the simulator. In this case, the camera's position and orientation may be selected to capture at least the pilot and a portion of the cockpit. In another example, the camera's position and orientation may be selected to capture only a given area or portion of the cockpit. In a further embodiment, the camera may be positioned outside the simulator to capture the area outside the simulator, allowing a third party to observe the simulator's movements.

[0082] In one embodiment, the camera's position and orientation are fixed in time. In another embodiment, the camera's position and / or orientation may change relative to the simulator as different views of the simulator can be captured. In this case, the camera's position and / or orientation may be controlled by a first computer machine.

[0083] While the above description refers to a single camera for capturing images of the simulator, it should be understood that multiple cameras may be used to capture images of the simulator. For example, several cameras may be installed inside the simulator to capture different views of the simulator's interior space. In this case, the position and orientation of each camera installed inside the simulator are chosen according to the respective desired views of the simulator's interior space. For example, a first camera may be positioned and oriented to capture images of the pilot, co-pilot, and cockpit, and a second camera may be positioned and oriented to capture an image of a given part of the cockpit. At least one additional camera may be installed outside the simulator to capture images of at least the exterior portion of the simulator.

[0084] When several views of the simulator are captured by several cameras, step 12 includes receiving video feeds from the cameras over a private network on a first computer machine. In one embodiment, a viewer application is configured to select one of the received video feeds and provide the selected video feed for display on a first display unit connected to the first computer machine. In another embodiment, a viewer application is configured to select at least two of the received video feeds, generate an image containing the two selected video feeds, and provide the generated image for display on a first display unit connected to the first computer machine. For example, a first video feed containing the internal portion of the simulator and a second video feed containing the external portion of the simulator may be received in step 12. Then, in step 14, the viewer application generates an image containing a first image from the first video feed and a second image from the second video feed. For example, half of the generated image may correspond to the image from the first video feed, and the other half of the generated image may correspond to the image from the second video feed. For example, images from two video feeds may be positioned side-by-side in the generated image, or one may be positioned on top of another. The image generated by the viewer application is then transmitted to a first display unit for display. Furthermore, simultaneously with the display of the generated image on the first display unit connected to the first computer machine, the generated image is transmitted via the telecommunications network by the first video conferencing client to the second video conferencing client, resulting in the display of the generated image on the first display unit connected to the first computer machine and the display of the same generated image on the second display unit connected to the second computer machine occurring substantially simultaneously.

[0085] For example, an image generated by a viewer application from three video feeds may contain three regions where each video feed is integrated. The upper left region of the generated image may be used to display a first video feed captured by a first camera, encompassing the pilot, co-pilot, and equipment, as well as the control panel. The upper right region of the generated image may be used to display a second video feed captured by a second camera, encompassing parts of the equipment and the control panel. The lower left region of the generated image may be used to display a third video feed captured by a third camera, encompassing a display unit present in the simulator, in order to display a computer-generated image of the simulated image, for example, a computer-generated image of the aircraft's external environment as seen through the windows of a real aircraft.

[0086] Figure 2 illustrates an exemplary pole 50 that may be used to detachably mount multiple cameras 51 within a simulator cockpit. The pole 50 includes an extension 52 that extends longitudinally between a first end fitted to the cockpit floor and a second end fitted to the cockpit ceiling. The length of the extension 52 is adjustable to accommodate cockpits of different heights. For example, the extension 52 may be telescopic. In one embodiment, rubber pads are mounted on the first and second ends of the extension 52, respectively, to improve the securing of the pole 50 into the cockpit. It should be understood that the extension 52 may have any suitable shape. For example, the extension 52 may have a circular cross-section, a square cross-section, and so on.

[0087] The pole 50 further includes a plurality of camera holding devices 54, each of which is configured to be mountable at a position along the length of the extension body 52 and to have one camera 51 secured thereto. Each camera holding device 54 includes a clamp 60 mountable on the pole 50 and an arm 62 extending between a first end 64 rotatably secured to the clamp 60 and a second end 66 rotatably secured to one of the cameras 51. Once mounted on the second end 66 of the arm 62, one of the cameras 51 is then rotatable relative to the arm 62. In one embodiment, the clamp 60 is slidably mounted on the extension body 52.

[0088] In one embodiment, a first ball joint is connected between the first end 64 of the arm 62 and the clamp 62 to allow the arm 62 to rotate around three axes of rotation relative to the clamp 60. Similarly, a second ball joint is present at the second end 66 of the arm 62 to secure a camera to the arm 62 and allow the camera to rotate around three axes of rotation relative to the arm 62.

[0089] To mount the pole 50 in the cockpit, the length of the extension 52 is adjusted to the height of the cockpit, i.e., the distance between the cockpit ceiling and the floor. The pole 50 is then positioned at a desired position on the floor such that its first end touches the cockpit floor and its second end touches the cockpit ceiling. Since the length of the pole 50 is substantially equal to the distance between the cockpit floor and the ceiling, a compressive force exists between the pole 50 and the cockpit floor and ceiling, thereby enabling the pole 50 to be secured in the cockpit. It should be understood that the pole 50 extends substantially vertically when secured in the cockpit.

[0090] Once the pole 50 is secured in the cockpit at the desired position on the floor, each of the camera holding devices 54 is positioned at its respective desired longitudinal position along the extension body 52. ​​Once in the desired longitudinal position, clamps 60 are used to secure each of the camera holding devices 54 to the extension body 52. ​​Each of the cameras 51 is then secured to its respective camera holding device 54. Alternatively, the cameras 51 may be secured to the camera holding devices 54 before positioning the camera holding devices 54 to their desired longitudinal positions.

[0091] Next, each camera 51 is oriented to capture an image of each desired view of the cockpit by rotating each camera 51 relative to its corresponding arm 62 and / or rotating its corresponding arm 62 relative to its corresponding clamp 60.

[0092] In embodiments where the elongated body 52 has a cylindrical shape, the clamp 60 is further rotatable around the longitudinal axis of the elongated body 52. ​​In this case, the clamp 60 can be positioned at a desired radial position along the circumference of the elongated body 52 before securing the clamp to the elongated body in order to properly orient each of the cameras 51.

[0093] It should be understood that the pole 50 can be detachably secured within the cockpit in any suitable position that allows the camera 51 to capture a desired view of the cockpit. In one embodiment, the pole 50 is secured in a suitable position within the cockpit that allows the pilot and co-pilot to access their respective seats and / or to easily leave the cockpit in an emergency.

[0094] In one embodiment, the pole 50 is secured within a simulated area of ​​the cockpit. For example, the pole 50 may be secured in front of the central platform of the cockpit, such as between the pilot and co-pilot seats. In another embodiment, the pole 50 is secured behind the pilot and co-pilot seats, facing the central platform.

[0095] In one embodiment, the pole 50 is mounted in a position within the cockpit, and as a result, the distance between each of the cameras 51 mounted on the pole 50 and the respective parts of the cockpit that will be imaged ranges from approximately 121.92 cm (4 feet) to approximately 185.04 cm (6 feet).

[0096] It should be understood that the number of camera holding devices 54, and therefore the number of cameras 51, may be variable insofar as the pole 50 includes at least one of the camera holding devices 54. In one embodiment, the pole 50 includes at least three of the camera holding devices 54. In this case, a first camera holding device 54 may be configured to have one of the cameras 51 mounted thereon to provide a first video feed encompassing the central pedestal of the cockpit. A second camera holding device 54 may be configured to have one of the cameras 51 mounted thereon to provide a second video feed encompassing the overhead equipment panel. A third camera holding device 54 may be configured to have one of the cameras 51 mounted thereon to provide a third video feed encompassing the pilot equipment panel. An additional camera holding device 54 may be provided so that an additional camera 51 provides a video feed of other parts of the cockpit, such as a video feed encompassing the co-pilot equipment panel, the pilot when in the pilot's seat, the co-pilot when in the co-pilot's seat, display units, etc.

[0097] In some embodiments, Method 10 further provides remote access to the simulator's simulation control interface, such as the interface of an instructor operating station. The operating system's simulation control interface facilitates the exchange of data with the simulation engine that generates the simulation. For example, the simulation control interface may be used to set and display parameters for the simulation. In one embodiment, a first computer machine communicates with the simulation engine over a private network, and the simulation engine directly transmits the simulation control interface to a viewer application. The viewer application combines the simulation control interface with a video feed received from a camera to display an image on a first display unit that includes both the video feed and the simulation control interface. For example, two adjacent areas of the generated image may be used to display the simulation control interface and the video feed, respectively.

[0098] As described above, the operating system interface can be received from the simulation engine by a viewer application, and other embodiments are possible. For example, the simulation control interface may be displayed on a further display unit separate from the first display unit, for example, on a display unit provided on an instructor station in the simulator, and a camera captures an image of the further display unit to provide a video feed encompassing the control interface. The video feed encompassing the simulation control interface is then sent to the viewer application, which generates an image using the video feed encompassing the simulation control interface and a video feed encompassing a portion of the simulator.

[0099] In one embodiment, images generated by the simulation engine may also be sent to the first computer machine, and as a result, the viewer application combines at least one video feed and, optionally, an operating system interface with them.

[0100] In one embodiment, a first computer machine is connected to a server hosting a data analysis platform over a private network. The data analysis platform remembers the user's historical interactions with the second computer machine, such as the amount of time the user spent on a given view and the location of user clicks on images displayed on a second display unit. In this case, method 10 further provides remote access to the interface of the data analysis platform.

[0101] In this embodiment, the first computer machine directly transmits the interface of the data analysis platform to the viewer application. The viewer application combines the interface of the data analysis platform with the video feed received from the camera to display an image on the first display unit that includes both the video feed and the interface of the data analysis platform.

[0102] As described above, the interface of the data analysis platform can be received directly from the server hosting the data analysis platform by a viewer application, and other embodiments are possible. For example, the interface of the data analysis platform may be displayed on a further display unit separate from the first display unit, and a further camera may capture images of the further display unit to provide a video feed encompassing the interface of the data analysis platform. The video feed encompassing the interface of the data analysis platform is then sent to the viewer application, which generates images using the video feed encompassing the interface of the data analysis platform and a video feed encompassing parts of the simulator.

[0103] Figure 3 illustrates an exemplary image resulting from a combination of six different images or video feeds executed by the viewer application. The exemplary image is divided into six equal sections, each used to display its respective image or video feed. In the upper right, an image of the instructor operating system interface is displayed. In the upper center, a video feed encompassing the upper part of the simulator is displayed. In the upper right, a simulated image of the aircraft's external environment is displayed. In the lower left, a video feed encompassing the external part of the simulator is displayed. In the lower center, a video feed encompassing the first part of the cockpit is displayed. In the lower right, a video feed encompassing the second part of the cockpit is displayed.

[0104] The image illustrated in Figure 3 is transmitted from the first video conferencing client to the second video conferencing client over a telecommunications network. Figure 4 illustrates an exemplary interface of the second video conferencing client as displayed on the second display unit of the second computer machine. The illustrated interface includes the image generated by the viewer application and transmitted to the second video conferencing client, and further includes icons below the image corresponding to each participant in the video conferencing session.

[0105] For example, an image generated by a viewer application may combine seven video feeds and a simulated image. The image generated by the viewer application may be divided into four areas. The upper left area may be used to display six different video feeds, each containing a different part of the simulator. The upper right area of ​​the image may be used to display a video feed containing a portion of the command and control panel. The lower right area of ​​the image may be used to display a simulated map with overlaid aircraft positions. The lower left area of ​​the image may be used to display an enlarged version of one of the six images displayed in the upper left. For example, by double-clicking on one of the six video feeds displayed in the upper left, the selected video feed will be enlarged and displayed in the lower left area.

[0106] In one embodiment, method 10 further includes transmitting an audio signal between a first video conferencing client and a second video conferencing client over a telecommunications network. In one embodiment, at least one microphone and / or at least one speaker may be present in the simulator to capture sound and / or emit an audio signal. The microphone and speaker communicate with a first computer machine. For example, the microphone and speaker may be connected to the first computer machine over a private network. In another example, Bluetooth® communication may be used between the first computer machine and the microphone and / or speaker.

[0107] When a microphone is positioned within the simulator, it captures sounds emitted within the simulator, and the audio feed is transmitted by the microphone to a first computer machine. The received audio feed is then transmitted by the first video conferencing client to a second video conferencing client over a telecommunications network during a video conferencing session. It should be understood that the second computer machine is equipped with at least one speaker to play back the received audio feed so that a third party can hear the sounds generated within the simulator.

[0108] Similarly, the second computer machine may be equipped with a microphone for capturing sound, and the corresponding audio feed is then transmitted to the first video conferencing client by the second video conferencing client via a telecommunications network. The first computer machine then receives the audio feed from the second computer machine and transmits the audio feed to a speaker present in the simulator.

[0109] Therefore, voice communication can be established in virtually real time between the pilot and / or co-pilot present in the simulator and a third party located remotely via a video conferencing session. The third party may transmit vocal commands or comments to the pilot and co-pilot while listening to any discussion between them, in addition to hearing background noise within the simulator.

[0110] In one embodiment, the pilot and co-pilot are each provided with their own headsets connected to a first computer machine. In another embodiment, at least one microphone may be provided within the simulator, such as between the pilot and co-pilot, to capture sound within the simulator, and at least one speaker may be provided within the simulator.

[0111] In one embodiment, the method 10 may provide visual communication between a third party and the pilot. For example, the third party may input commands / requests / instructions, such as text, images, or diagrams, into a second computer device. The commands are transmitted from the second video conferencing client to the first video conferencing client over a telecommunications network. Upon receiving the commands, the first computer machine transmits the received commands to a display unit located within the simulator to notify the pilot and / or co-pilot of the commands. For example, the commands may be displayed on a display unit included in the simulator's equipment and control panel. In another embodiment, the display unit may include a projector configured to project commands within the simulator for notification to the pilot and / or co-pilot.

[0112] In one embodiment, a viewer application running on a first computer machine may be remotely controlled by a second computer machine via a video conferencing platform. In this case, method 10 further includes the step of allowing the second video conferencing client to control the viewer application via the first video conferencing client. In this case, a third party may input commands within the second video conferencing client. In this way, the third party may interact with, for example, the interface of a simulator's instructor-operator station. When input is input to the second video conferencing client on the second computer machine, the input is transmitted from the second video conferencing client to the first video conferencing client over the telecommunications network. The input is then received by a viewer application configured to execute commands. For example, when it detects a mouse click, the second video conferencing client may determine the position of a cursor in the image displayed by the second video conferencing client. The second video conferencing client then sends the cursor position to the first video conferencing client, which sends the received cursor position to the viewer application. The viewer application interprets the received cursor as a mouse click at the same position in the image generated by the viewer application. For example, if the cursor position corresponds to a button on the interface of the instructor-operator station, the viewer activates the button, i.e., it sends a command to the simulation engine indicating that the identified button should be activated. A third party may interact with the interface of the data analysis platform.

[0113] In one embodiment, where the first and second video conferencing clients are part of a pre-built video conferencing platform, existing functions such as “Give control” and “Take control” may be used to allow the second video conferencing client to control the viewer application. The second video conferencing client can then be granted control over the operating system interface and / or the data analytics platform interface.

[0114] It should be understood that Method 10 described above is embodied as a system that provides remote visibility to a simulation environment, the system comprising at least one processor and a non-temporary storage medium operablely connected to the processor, the non-temporary storage medium containing computer-readable instructions, and the processor, upon executing a computer-readable instruction, is configured to perform the steps of Method 10.

[0115] Method 10 described above may be embodied as a computer program product that provides remote visibility into a simulation environment, and it should also be understood that the computer program product includes computer-readable memory that stores computer-executable instructions therein, which, when executed by a processor, perform the method steps of Method 10.

[0116] Figure 5 illustrates one embodiment of system 100 that provides remote visibility to the simulation environment. System 100 can be used to perform method 10.

[0117] System 100 includes a simulation environment or simulator 102, a private location 102, and a remote location 104. Simulator 102 includes at least equipment and controls (not shown) with which the user of simulator 102 interacts to run the simulation, at least one camera 110 for capturing images of at least a portion of simulator 102, including at least a portion of the equipment and controls, at least one speaker 112 located within simulator 102, and at least one microphone 104 located within simulator 102.

[0118] Private location 104 includes a first computer machine 120 and a first display unit 122 connected thereto. A viewer application and a first video conferencing client are running simultaneously on the first computer machine. The first computer machine 120 communicates with the camera 110, speaker 112, and microphone 114 of the simulator 102. The camera 110 is connected to the computer machine 120 via a private network. In one embodiment, the speaker 112 and microphone 114 are also connected to the computer machine 120 via a private network. In another embodiment, wireless communication, such as Bluetooth® communication, may be used to connect the first computer machine and the speaker 112 and microphone 114 as described above. In this case, it should be understood that private location 122 is located in close proximity to the simulator to enable wireless communication between the speaker 112 and microphone 114 and the first computer machine 120.

[0119] The remote location includes a second computer machine 130, a second display unit 132, a speaker 134, and a microphone 136. The second video conferencing client runs on the second computer machine 130. Data can be exchanged via the telecommunications network 138 between the first video conferencing client running on the first computer machine 120 and the second video conferencing client running on the second computer machine 130. It should be understood that the second computer machine 130 does not have access to the private network 124.

[0120] As described above, a viewer application running on the first computer machine 120 receives a video feed from the camera 110 over the private network 124 and provides the video feed for display on the first display unit 122. Furthermore, the viewer application sends the video feed to the first video conferencing client, and the first video conferencing client sends the video feed to the second video conferencing client over the telecommunications network 138.

[0121] A second video conferencing client, running on a second computer machine 130, receives a video feed from the first video conferencing client and transmits the received video feed for display on the second display unit 132.

[0122] Speaker 112 and microphone 114, and speaker 134 and microphone 136 enable voice communication between the simulator environment 102 and the remote location 106, as described above. Any voice signal captured by microphone 114 is transmitted to the first computer machine 1210 via the first and second video conferencing clients over the telecommunications network 138, and then to the second computer machine 130. The voice signal is then reproduced by speaker 134. Similarly, any voice signal captured by microphone 136 is transmitted to the first computer machine 120 via the telecommunications network 138, over the first and second video conferencing clients. The first computer machine 120 then transmits the received voice signal to speaker 112, which reproduces the voice signal.

[0123] In one embodiment, the system 100 further includes a simulation engine 140 that generates a simulation for a simulator 102. The simulation engine 140 communicates with a first computer machine 120 via a private network 124. The simulation engine 140 is configured to transmit an operating system interface to the first computer machine 120 over the private network 124. A viewer application running on the first computer machine 120 receives the interface of the simulation engine 140 and generates an image including the video feed received from the camera 110 and the operating system interface. The viewer application then transmits the generated image to a first display unit 122 and a first video conferencing client, which will be displayed thereon, and the first video conferencing client transmits the image to a second video conferencing client as described above.

[0124] In one embodiment, the system 100 further includes a data analysis platform hosted on a server 142 connected to a private network. The server 142 is configured to transmit the interface of the data analysis platform to a first computer machine 120 over the private network 124. A viewer application running on the first computer machine 120 receives the interface of the data analysis platform and generates an image including the video feed received from the camera 110 and the interface of the data analysis platform. The viewer application then transmits the generated image to the first display unit 122 and the first video conferencing client, which will be displayed thereon, and the first video conferencing client transmits the image to the second video conferencing client as described above.

[0125] As described above, commands / requests can be transmitted from the second computer machine 130 to the first computer machine 120 via the first and second video conferencing clients over the telecommunications network 138. For example, a predefined input entered into the second computer machine 130 may be recognized as a command and sent to the viewer application on the first computer machine 120. For example, a mouse click or action of a given key on a user interface connected to the second computer machine 130 may be identified as a command. In this case, the second video conferencing client sends the command to the first video conferencing client, and the viewer application executes the command.

[0126] As described above, in embodiments where the first and second video conferencing clients are part of a ready-made video conferencing platform, control of the viewer application may be permitted to the second computer device using predefined functions of the ready-made video conferencing platform.

[0127] Figure 6 illustrates an exemplary cockpit of an aircraft simulator. The simulator is: The main equipment panel includes a pilot equipment panel and a co-pilot equipment panel, A central platform positioned between the pilot and co-pilot seats, Overhead equipment panel and At least one display unit for displaying a simulation image thereon, Two front cameras positioned in front of the pilot and co-pilot, Three rear cameras are positioned behind the pilot and co-pilot, each oriented to capture a full view of the pilot's instrument panel, the co-pilot's instrument panel, and the entire cockpit. An upward-facing camera positioned and oriented to image the overhead equipment panel, A downward-facing camera positioned and oriented to image the central pedestal, An external camera for imaging the external parts of the simulator, Two projectors positioned behind the pilot and co-pilot to display images from inside the cockpit, Two microphones, each positioned on the opposite side of the main equipment panel, The central microphone is positioned on the base, Two speakers, each positioned on the opposite side of the main unit panel, This includes a central speaker positioned on a base.

[0128] The camera, microphone and speaker, and projector all communicate with the first computer machine. The video feed captured by the camera is received by a viewer application that generates an image combining the received video feeds, and the generated image is transmitted by the first video conferencing client to the second video conferencing client and to the first display unit. The audio feed received from the microphone is also transmitted by the first video conferencing client to the second video conferencing client. Any audio feed received by the first video conferencing client from the second video conferencing client is transmitted to the speaker and played back. Any visual commands, such as text or images, received by the first video conferencing client from the second video conferencing client are transmitted to at least one of the two projectors and displayed in the cockpit. Projectors may be used to display images on different equipment panels to point to specific equipment, as will be described in more detail below.

[0129] In one embodiment, the pilot and co-pilot may each be provided with a headset connected to a first computer machine. In this case, the speaker and microphone described above may be omitted.

[0130] It should be understood that some of the components of the simulator illustrated in Figure 6 may be omitted, or other components may be added. Similarly, it should be understood that the positions and orientations of the components are illustrative only.

[0131] Figure 7 illustrates an exemplary visual interface generated by a second video conferencing client. Using this interface, a user of the second computer machine can select the video feeds and images of the interface that the user wishes to view. In the upper left of the visual interface, the user can select the layout of the displayed images. For example, the user may choose to display a single video feed or interface image, two different video feeds or interface images positioned side by side, four video feeds or interface images, or six video feeds or interface images.

[0132] The top control bar includes boxes indicating which video feeds and interface images are available. In this example, five different video feeds and four different interface images are available. In the example shown, the layout includes four windows, so the user can select four items from the available video feeds and interface images, each of which will be displayed in a separate window.

[0133] Figure 8 illustrates an exemplary setup when four display units are connected to a second computer machine. In this embodiment, each display unit is provided with the same visual interface by the second video conferencing client. The user may select different layouts for different display units. In the exemplary embodiment, three display units display interfaces containing a single window, each displaying a single video feed or a single interface, while a fourth display unit displays an interface containing four windows, each displaying four different video feeds or interfaces.

[0134] The above methods and systems are described in the context of flight simulators or aircraft simulators, and it should be understood that they may be used for other types of simulators.

[0135] In embodiments where an integrated camera, microphone, and speaker are already provided in the simulator, the existing camera, microphone, and speaker may be used to implement the technology.

[0136] In one embodiment, the camera used to image at least a portion of the simulator is an Internet Protocol (IP) camera compatible with streaming of the captured images. In one embodiment, the camera is a low-light-sensitive color camera. In one embodiment, the camera is provided with a zoom for adjusting its field of view. In this case, the field of view may be controlled from a first computer machine or a second computer machine. In one embodiment, the camera can be mounted so that it does not move with the buffet-up to 3G. In one embodiment, the camera supports a minimum resolution of 1080 pixels and 30 frames per second, and / or supports RTSP (Streaming Protocol) and H.264 compression.

[0137] In one embodiment, the camera is a Panasonic® WV-S1131. In another embodiment, the camera is a miniature camera such as an Axis camera. For example, the camera may be a pen camera or pinhole camera with a wide field of view, such as better than 90 degrees and good low-light characteristics. Such a miniature camera may be used when a larger camera would be cumbersome or too intrusive.

[0138] In one embodiment, the camera is mounted on a custom-made or custom-assembled bracket specific to each cockpit to maximize the view while minimizing the level of intrusion. When it is a miniature camera, the camera may be mounted either within the main equipment panel (MIP) or within the glare shield control panel (GCP).

[0139] In one embodiment, a dedicated segment is created within a private network to prevent the operation of this system from affecting the overall operation of the simulator.

[0140] In one embodiment, a virtual private network (VPN) is used to communicate with the simulator over a wide area network (WAN). The VPN enables tunneling through various network firewalls / routers / switches to ensure good performance. Password protection is provided through the standard VPN process. In one embodiment, encryption or VPN may not be present when the system is running on a local area network (LAN).

[0141] In one embodiment, the camera is connected to an onboard router to access a dedicated segment of the simulator network. A first computer machine is connected to the segment to receive a video feed from the camera.

[0142] In one embodiment, the first computer machine has simultaneous access to the simulator network and the internet for managing bidirectional communication between the simulator and the second computer machine. The first computer machine acts as a bridge between the internet and the simulator network, and the bridge prevents the second computer machine from having direct access to the simulator network. The second computer machine may belong to, for example, a client or a regulating entity.

[0143] In one embodiment, an application such as a viewer application may run locally on the processor of a first computer machine, while a first video conferencing client may run at least partially in the cloud. In another embodiment, the application may run primarily in the cloud.

[0144] In one embodiment, the first computer machine is located onboard and enables a wired connection to the conference speaker. In one embodiment, the first computer machine can be installed in the same location as the simulator, for example, when the first computer machine is located adjacent to the simulator, such as when the first computer machine and the simulator are located in the same building, or when the first computer machine is located remotely from the simulator, such as when the first computer machine and the simulator are located in different buildings.

[0145] The first and second computer machines may be any devices equipped with at least one processor, memory, and communication means. For example, the first or second computer machine may be a laptop, desktop, tablet, smartphone, etc.

[0146] In one embodiment, the management of audio within the system described above can be carried out in three ways: 1) The ambient sounds of the cockpit generated by the simulation, such as engine noise, external airflow, aerodynamic hiss, and mechanical noises such as gear and flap deployment, 2) For example, radio communication that simulates conversations between air traffic control and aircraft crew, 3) Take into account crew-instructor communication, which is a natural part of the training process.

[0147] In embodiments where the pilot and / or co-pilot are provided with a headset, the pilot or co-pilot hears communications and mutual communications through the headset, and ambient sounds are generated by a simulator sound system. The pilot's voice is captured by the headset microphone.

[0148] In embodiments where a headset is provided to the pilot and / or co-pilot, a special microphone is mounted at the key position, along with an additional speaker that may be connected to a digital signal processing system, to eliminate echoes and ambient noise. Simulated aircraft sounds are generated by a simulator sound system.

[0149] In one embodiment, the following six views are shown: The pilot's main equipment panel, The co-pilot's main equipment panel, The central pedestal and Overhead panel and, A close-up of one of the primary flight displays, At least six different cameras are used to capture one each of the two pilots' views, including a view from behind them with the instructor leaning forward, and a view from behind their shoulders. Such views provide an overall wide angle encompassing the equipment, out-of-window vision, and the pilots.

[0150] In one embodiment, a camera discreetly mounted within the main equipment panel facing the rear provides a view of the pilot's face and reactions.

[0151] In one embodiment, the data analysis platform is hosted in the cloud. Analysis data can either be downloaded to a simulator for viewing, such as to adapt to an instance simulator due to limited internet connectivity, or the analysis data can be viewed directly from the cloud.

[0152] The following describes a method and system for remotely operating at least one projector to point to a region of interest within a simulator. In one embodiment, the method described below may be used in conjunction with the system 100 described above to enable a user, such as the third party described above, to remotely operate the projector to point to a region of interest within the simulator via a video conferencing platform.

[0153] The methods and systems described below attempt to replicate some of the interactions that occur between an instructor and a pilot / co-pilot when the instructor is physically present in the simulator with the pilot / co-pilot. During such training, when the instructor is physically present in the simulator, the instructor is typically required to point to specific aspects of aircraft system operation and / or content on one or more multifunction displays.

[0154] The conventional method adopted requires the instructor to move forward from a seated position, secured by a seatbelt as a safety precaution for aircraft simulators equipped with motion systems, toward the pilot / co-pilot or a designated point. However, under certain circumstances, such as during a pandemic, it may be advantageous for the instructor to be positioned remotely, i.e., outside the simulator, for example, to reduce the risk of contamination.

[0155] Figure 9 illustrates one embodiment of Method 150 for remotely pointing to a region of interest within a simulator or simulation environment. It should be understood that Method 150 is performed by a computer machine equipped with at least one processor, at least one memory, and a communication interface. Method 150 enables a user, such as an instructor, who is not present in the simulator, to visually interact with a human being located within the simulator, such as a pilot or co-pilot, by remotely pointing to a given region of interest within the simulator. As described below, Method 150 can be performed over a secure public network via a video conferencing platform.

[0156] In step 152, images of at least a portion of the simulator are provided by the first computer machine for display on a remotely located display unit. The images represent at least a portion of the interior of the simulator, such as at least a portion of the cockpit and at least a portion of the equipment and controls.

[0157] In one embodiment, the image is captured by at least one camera installed within the simulator. For example, a single camera that is movable within the simulator may be used to capture a view of the inside of the simulator. In another embodiment, a single camera having a fixed position and / or orientation may be used to capture a view of the inside of the simulator. In a further embodiment, multiple cameras, each having a fixed position and orientation, may be installed within the simulator. The image provided for display in step 150 may correspond to an image captured by a given camera. In another embodiment, the image may result from a combination of multiple images captured by multiple cameras. In this case, method 150 further includes the step of combining multiple images received together from different cameras to obtain a single image.

[0158] In another embodiment, the image provided for display in step 152 may be a simulated image or a computer-generated image of at least a portion of the simulator's interior. In one embodiment, the image may be a three-dimensional (3D) image of at least a portion of the simulator. In this case, method 150 further includes the step of generating an image of at least a portion of the simulator's interior.

[0159] Images are transmitted over a communication network by a first computer machine to a second or remote computer machine located outside the simulator. For example, the communication network may be a secure private network. In another example, the communication network may be a public network such as the Internet. The remote computer machine is connected to a user interface for entering commands into the remote computer machine and a display unit for displaying the received images from the simulator on it.

[0160] When the simulator displays the received image, the user of the remote computer machine may select at least one point on the displayed image using any suitable user interface, such as a mouse, keyboard, or stylus, when the display unit includes a touchscreen.

[0161] In one embodiment, a single point is selected on the image displayed by the simulator. In another embodiment, multiple points are selected by the user. The multiple points can be discrete, that is, the selected points can be spaced apart from each other. Alternatively, the points can be adjacent to each other to form a continuous geometric shape such as a line, circle, or square.

[0162] It should be understood that any suitable method may be used to select at least one point on the image. For example, a mouse may be used to move the cursor over the desired points, a right or left click on the mouse may be used to produce a click event and select the desired points, or a key on the keyboard may be pressed to select the desired points. In another example, multiple points may be selected by right or left clicks on the mouse while moving the mouse over the desired points. In a further example, a user of a remote computer machine may drag across the displayed image to select one or more points.

[0163] The coordinates of the selected points in the displayed image are captured by the remote computer machine and transmitted to the first computer machine over the communication network.

[0164] It should be understood that when the remotely displayed image is a 2D image, the captured coordinates are 2D coordinates, and when the remotely displayed image is a 3D image, the captured coordinates are 3D coordinates.

[0165] In step 154, the position of the selected point in the remotely displayed image is received; that is, the coordinates of the selected point are received by the first computer machine over the communication network.

[0166] In one embodiment, the first and remote computer machines are connected via a public secure network, with the first computer machine having a first video conferencing client and the second computer machine having a second video conferencing client. In this case, the coordinates of the selected point are transmitted to the first computer machine via the second video conferencing client and received by the first computer machine via the first video conferencing client.

[0167] In step 156, the region of interest located on the simulator is identified based on the received coordinates of the selected point. For example, if a user of a remote computer machine selects a given button or actuator displayed on the image, step 156 involves identifying the region of interest on the simulator that includes the selected button or actuator.

[0168] In one embodiment, the identification of a region of interest is performed based on a mapping between the coordinate system of a remotely displayed image and the coordinate system of the simulator.

[0169] In one embodiment, the region of interest has a predefined shape and / or dimensions. In this case, step 156 includes identifying the position of the region of interest on the simulator based on the received coordinates. For example, if the coordinates of a single point are received in step 154, step 156 includes identifying the position of the region of interest on the simulator based on the received coordinates of a single selected point, and assigning a predefined shape and predefined dimensions to the region of interest. For example, the region of interest may have the shape of a square with predefined side lengths.

[0170] In embodiments where the coordinates of more than one point are received, the relative size of the region of interest may be greater than the relative size of the geometric object formed by the selected points. For example, if the selected points form a circle with a given diameter, the region of interest may also be circular in shape, and the diameter of the circular region of interest may be greater than the diameter of the circle formed by the selected points.

[0171] In embodiments where the coordinates of more than one selected point are received in step 154, the position of the region of interest may be identified based on the coordinates of at least one of the selected points, and the shape and dimensions of the region of interest may be determined based on the coordinates of the selected points. For example, if a user of a second computer machine draws a figure to select points, the region of interest may have the same shape as the figure, the dimensions of the region of interest may be selected based on the dimensions of the figure, and the position of the region of interest in the simulator may be identified based on the position of the figure in the displayed image, i.e., based on the coordinates of at least one point of the figure in the displayed image.

[0172] In one embodiment, a first coordinate system is assigned to the interior of the simulator, and a second coordinate system is assigned to the displayed image. The position of the region of interest is determined using a mapping between the first and second coordinate systems, including the received coordinates and a transfer function between the first and second coordinate systems. The mapping between the coordinate systems makes it possible to map a given point in the displayed image to the corresponding point in the simulator, i.e., to determine the coordinates of each point in the first coordinate system based on the coordinates of a given point in the second coordinate system.

[0173] In one embodiment, the method further includes a calibration step of determining the mapping between a first coordinate system and a second coordinate system.

[0174] Once the position of the region of interest in the simulator is identified in step 156, the region of interest in the simulator is visually distinguished in step 158, for example, the region of interest is illuminated. It should be understood that any suitable method for visually distinguishing the region of interest may be used. The simulator may be provided with at least one light source connected to a first computer machine that controls the light source. For example, the simulator may be provided with a single light source, and the position and / or orientation of the single light source is adjusted to illuminate the region of interest on the simulator. In another embodiment, the simulator may be provided with multiple light sources, and the method further includes the step of selecting a given light source that will be operated to illuminate the region of interest based on the position of the region of interest. In embodiments in which the simulator is provided with multiple light sources, the position and / or orientation of the light sources may be fixed. Alternatively, the position and / or orientation of at least one light source may be adjustable.

[0175] While using Method 150, users such as instructors may be positioned remotely from the simulator and still have the ability to visually interact with simulator users such as pilots or co-pilots. For example, by pointing a cursor at a given location on the simulator's displayed image, the corresponding location within the simulator is illuminated, thereby enabling the instructor to visually indicate the location within the simulator of controls, commands, buttons, etc., that will be activated during the simulation. It should be understood that Method 150 is performed substantially in real time, and as a result, any selected area or point on the displayed image is illuminated within the simulator substantially in real time.

[0176] In one embodiment, method 150 further enables a user of a second computer machine to talk with a user of a simulator. In this case, the second computer machine is connected to a microphone, and the simulator is provided with at least one speaker. Alternatively, the user of the simulator may be provided with a headset. The user of the second computer machine can then talk into the microphone, and the resulting live audio feed is sent to the first computer machine. In embodiments where the first and second computer machines are connected via a video conferencing platform, the live audio feed is transmitted by the second computer machine through the second video conferencing client and received by the first computer machine through the first video conferencing client. The first computer machine then transmits the received live audio feed to a speaker, which plays the audio feed in substantially real time.

[0177] Figure 10 illustrates a further embodiment of Method 200 for remotely pointing to a region of interest within a simulator. Method 200 corresponds to a specific implementation of Method 150 in which a projector is used to illuminate the region of interest within the simulator.

[0178] In step 202, as in step 152, images of at least a portion of the simulator are provided to the second computer machine by the first computer machine for remote viewing, as described above.

[0179] In step 204, as in step 154, the coordinates of at least one selected point in the displayed image are received by the first computer machine, as described above.

[0180] In step 206, the received coordinates are mapped to a pointer area within the projector's field of view. The pointer area corresponds to a portion of the projector's field of view and is defined by its position, shape, and size within the projector's field of view. In step 206, the position of the pointer area within the projector's field of view is determined based on the received coordinates.

[0181] In one embodiment, as described above with respect to Method 150, the pointer area has a predefined shape and / or dimensions.

[0182] In step 204, in embodiments where coordinates of more than one selected point are received, as described above with respect to method 150, the position of the pointer area within the projector's field of view is identified based on the coordinates of at least one of the selected points, and the shape and dimensions of the area of ​​interest may be determined based on the coordinates of the selected point.

[0183] In one embodiment, a first coordinate system is assigned to the projector or the projector's field of view, and a second coordinate system is assigned to the displayed image. The position of the pointer area in the first coordinate system is determined based on the received coordinates using a mapping between the first and second coordinate systems, such as a transfer function between the first and second coordinate systems. The mapping between the coordinate systems allows for determining the coordinates of each point in the first coordinate system based on the coordinates of a given point in the second coordinate system. The position of the pointer area in the projector's field of view is then selected based on the position of the point in the first coordinate system that corresponds to the selected point.

[0184] In one embodiment, the projector's coordinate system is mapped to the simulator's coordinate system, and as a result, at least one given point in the simulator's coordinate system corresponding to a selected point in the displayed image is first determined using the mapping between the simulator's coordinate system and the displayed image. The position of the pointer area in the projector's coordinate system is then identified by mapping the identified given point in the simulator's coordinate system to the projector's coordinate system.

[0185] In one embodiment, the method further includes a calibration step of determining the mapping between a first coordinate system and a second coordinate system.

[0186] In embodiments where the displayed image corresponds to a portion of the simulator encompassed by the projector's field of view, the coordinate system associated with the projector's field of view may coincide with the coordinate system of the displayed image, and consequently, a point having coordinates (x,y) in the displayed image also has coordinates (x,y) in the projector's field of view. In embodiments where the displayed image is a simulated image of the simulator, the displayed image may coincide with a portion of the simulator encompassed by the projector's field of view.

[0187] In embodiments where there is a small difference between the displayed image and the portion of the simulator encompassed by the projector's field of view, such that the coordinate system associated with the projector's field of view does not coincide with the coordinate system of the displayed image, the coordinates of a point in the projector's field of view may still be considered to coincide with its corresponding point in the displayed image, and the size of the pointer area may increase. In this case, the illuminated region of interest may become larger to ensure that the simulator point corresponding to the selected point is included within the region of interest. This may occur when the displayed image is an image captured by a camera located in the simulator, and the camera is positioned adjacent to the projector and has substantially the same orientation and field of view as those of the projector.

[0188] Once the pointer area is identified, the next step 208 involves generating an overlay image based on the pointer area.

[0189] In one embodiment, the overlay image corresponds to a pointer area, and as a result, the projector projects light only within the pointer area, and no light is projected in the portion of the projector's field of view outside the pointer area.

[0190] In another embodiment, the overlay image is larger than the pointer area. In this case, the overlay image includes a first and a second region. The first region corresponds to the pointer area, and the second region corresponds to the rest of the overlay image outside the pointer area. In this case, the first and second regions have at least one distinct feature to visually distinguish the pointer area from the rest of the overlay image. For example, the color of light generated within the pointer area may be different from the color of light generated for the rest of the overlay image. In the same or a different embodiment, the intensity of light may be different. For example, the intensity of light within the pointer area may be set to zero, while the intensity of the rest of the overlay image may be greater than zero, resulting in the region of interest being visually distinguished by not being illuminated, and the area surrounding the region of interest being illuminated. In another example, the intensity of light within the pointer area may be greater than zero, while the remaining intensity of the overlay image may be set to zero. As a result, the region of interest is visually distinguished by being illuminated, while the area surrounding the region of interest is not illuminated.

[0191] In step 210, the projector is controlled to project an overlay image onto the simulator; that is, a control signal indicating the overlay image is provided to the projector, thereby visually distinguishing the region of interest of the simulator. In one embodiment, the simulator illuminated by the light projected within the pointer area of ​​the projector's field of view forms the region of interest of the simulator.

[0192] In embodiments where the overlay image includes a single area corresponding to a pointer area, the projector projects light only within the pointer area identified within its field of view. Figures 11 and 12 illustrate such scenarios. Figure 11 illustrates an exemplary remotely displayed image of a portion of the cockpit. A square-shaped, orange-colored cursor is displayed over the cockpit image. A user of a second computer machine can move the square cursor to select an area of ​​interest that will be highlighted on the simulator's cockpit. The position of the square cursor is transmitted to the first computer machine, which determines the position of the pointer area in the projector's field of view based on the received position of the square cursor in the displayed image. The projector then projects an overlay image corresponding to the pointer area. The overlay image is given a square shape and orange color. Figure 12 illustrates the cockpit when the overlay image is projected over the cockpit. The orange square is then projected onto the cockpit at a position on the cockpit that is substantially the same as the position of the square cursor on the portion of the cockpit represented in the displayed image.

[0193] In another embodiment, where only a given area of ​​the overlay image corresponds to the pointer area, the projector projects an image onto the simulator, and the region of interest corresponds to the portion of the simulator illuminated by the given area. To visually distinguish the region of interest, as described above, the light emitted within the pointer area may be of a first color, such as orange, while the light emitted corresponding to a second area of ​​the overlay image may have a second and different color, such as white.

[0194] The size of the overlay image can be chosen so that it occupies the entire field of view of the projector, and then the pointer area corresponds to a portion of the field of view.

[0195] In one embodiment, methods 150, 200 are embodied as a system including a processor and a non-temporary storage medium operably connected to the processor, the non-temporary storage medium containing computer-readable instructions, and the processor is configured to execute the steps of methods 150, 200 when it has executed a computer-readable instruction.

[0196] In another embodiment, methods 150, 200 are embodied as a computer program product that includes computer-readable memory storing computer-executable instructions therein, which, when executed by a processor, perform the steps of methods 150, 200.

[0197] Figure 13 illustrates one embodiment of System 250 that allows a user to remotely point to a region of interest within the simulator.

[0198] System 250 includes a simulation environment or simulator 252, a private location 253, and a remote location 254. Simulator 252 includes at least equipment and controls (not shown) with which a user of simulator 152 interacts to run the simulation, and at least one projector 260. The simulator may further include at least one camera 262 for capturing images of at least a portion of simulator 252, such as at least a portion of the equipment and controls, at least one speaker 266 located within simulator 252, and at least one microphone 264 located within simulator 252.

[0199] Private location 253 includes a first computer machine 270 configured to perform the steps of method 200. The first computer machine 270 is also provided with a first video conferencing client. The first computer machine 270 communicates with the projector 260 via a private secure network 274. The first computer machine 270 may also communicate with a camera 262, a speaker 266, and / or a microphone 264 if it is located within the simulator 252.

[0200] The remote location 254 includes a second computer machine 280, a second display unit 282, and a user interface 284. It should be understood that the user interface 284 may be omitted if the second display unit 282 includes, for example, a touchscreen. The remote location 254 may optionally include a speaker 288 and a microphone 286. The second video conferencing client is located on the second computer machine 280. Data can be exchanged via the public secure network 290 between the first video conferencing client running on the first computer machine 270 and the second video conferencing client running on the second computer machine 280. It should be understood that the second computer machine 280 does not have access to the private network 274.

[0201] In one embodiment, the system 250 includes an image generator 292 for generating images of at least a portion of the simulator 252, such as a cockpit. The image generator 292 may be configured to generate images from a 3D model of the simulator 252. In another embodiment, the simulator 252 includes a camera 262 for capturing live images of at least a portion of the simulator 252.

[0202] The image is transmitted to the first computer machine 270 over the secure private network 274. The first computer machine 270 transmits the image of the simulator 252 to the second computer machine 280 via the first video conferencing client over the secure public network 290. The second computer machine 280 receives the image of the simulator 252 via the second video conferencing client and provides the received image to the display unit 282 for display. A user of the second computer machine 280, such as an instructor, can then select at least one point in the displayed image of the simulator 252. For example, if the user interface 284 includes a mouse, the user can left-click on a given command displayed in the image to provide the user of the simulator 252 with a visual indication of the location of a given command in the simulator 252.

[0203] The second computer machine 280 captures the coordinates of the selected point and transmits the captured coordinates to the first computer machine 270 via the second video conferencing client over the secure public network 290. The first computer machine 270 then receives the coordinates of the selected point via the first video conferencing client and maps the received coordinates to the pointer area in the field of view of the projector 260 before generating an overlay image based on the pointer area as described above. The first computer machine 270 transmits the overlay image to the projector 260, which projects the overlay image onto the simulator 252 to illuminate the area of ​​interest as described above.

[0204] In one embodiment, the displayed image is static. In another embodiment, the displayed image is dynamic and may change over time.

[0205] In one embodiment, speakers 266 and 288 and microphones 264 and 286 enable voice communication between the simulator 252 and a remote location, so that the instructor and pilot can, for example, talk during a training simulation. In this case, the live audio feed captured by microphone 286 is transmitted to the first computer machine 270 via a second video conferencing client over a secure public network 290. The first computer machine 270 receives the live audio feed via the first video conferencing client and transmits the live audio feed to speaker 266, which will be played back in the simulator 252. Similarly, microphone 264 captures the live audio feed from the simulator 252 and transmits the captured live audio feed to the first computer machine 270 via a secure private network 274. The first computer machine 270 then transmits the live audio feed to the second computer machine 280 via a first video conferencing client over the secure public network 290. The second computer machine 280 receives a live audio feed through the second video conferencing client and transmits the live audio feed to the speaker 288, which is to be played back.

[0206] In embodiments where the image displayed in the simulator is a live video feed captured by a camera located within the simulator, the camera's orientation, position, and / or zoom may be adjusted so that the camera can capture a desired view of the simulator.

[0207] In an embodiment in which the simulator is provided with multiple cameras, each positioned and oriented to capture each view of the simulator, a desired view of the simulator is obtained by selecting a given camera.

[0208] In one embodiment, a view may be selected remotely from a second computer machine, for example, via a video conferencing platform, when control is granted to the camera. The user of the second computer machine may then remotely control the position, orientation, and / or zoom of the camera located within the simulator and / or select a given camera to obtain a desired view of the simulator.

[0209] Although the above description refers to a single projector, such as projector 260, it should be understood that a simulator may be equipped with multiple projectors. Each of the multiple projectors may cover a different area of ​​the simulator. In this case, method 200 further includes the step of selecting a given projector based on the coordinates of a selected point, such that the area of ​​interest to be distinguished is located within the field of view of the given projector.

[0210] In embodiments where a single projector is used, the projector has a fixed position and orientation. In another embodiment, the position and / or orientation of the projector is adjustable. In this case, method 200 may further include the steps of determining the required movement, such as translation or rotation, required for the projector when the area of ​​interest is not within the projector's field of view, and moving the projector according to the determined required movement.

[0211] In one embodiment, at least one feature of the region of interest or pointer area may be selected by the user of the second computer machine. For example, the shape, size, and / or color of the region of interest or pointer area may be selected. For example, the region of interest or pointer area may have a shape such as a circle, an arrow, or an "X".

[0212] In embodiments where a projector is used, the image projected by the projector may be animated.

[0213] In one embodiment, the method further includes the step of determining mapping parameters to ensure alignment between the projected image and the cockpit equipment and controls. In this case, the step of mapping coordinates includes the step of applying the mapping parameters to coordinates to determine the pointer area in the overlay image.

[0214] In one embodiment, the parameters are determined as a function of the projector position, the optical characteristics of the projector, and the geometric shape of the cockpit structure.

[0215] In one embodiment, the parameters are determined by projecting a series of features and identifying the location of the features on the simulator.

[0216] In one embodiment, the identification of feature locations is performed automatically through image processing means.

[0217] In one embodiment, the projector includes at least one of the following characteristics: (a) Since power may not be easily accessible within the cockpit area, the projector shall have Power-Over-Ethernet (PoE) functionality. (b) The projector has built-in capabilities to run client applications, (c) The projector generates low heat and minimal fan noise.

[0218] In one embodiment, the projector includes a lens that provides the required cockpit coverage, HD resolution, and brightness.

[0219] In one embodiment, in an offboard IOS, there is a dedicated monitor for the pointer system, including a server application. The server application interfaces directly with each client running on each projector. The server application includes UI management and an interface with the instructor. The projector / view is selected in IOS and the corresponding static image displayed. As the cursor moves to the desired position, positioning data is transferred to the projector / client application. The actual cursor is visible to the instructor using a camera feed returning from the cockpit. The server application may include different choices for at least some characteristics of the cursor, such as cursor symbol type, size, and color.

[0220] In one embodiment, the server application is expanded to capture a predefined area of ​​interest that demonstrates usage for the purpose of product usage analysis.

[0221] In one embodiment, the use of a movable camera delivering (3D) dynamic images may allow for a reduction in the number of projectors required. A 3D model of the cockpit area is required, along with additional logic, to move virtually within the 3D world as commanded by the instructor, and to display pointers. This can be achieved by creating a mapping matrix between camera orientation (azimuth, elevation) and simulator coordinates. This can be done manually or through a semi-automatic procedure, in which the camera displays a test pattern (e.g., grid) swivel through its range, stopping at each grid interval. The user is then prompted to confirm the grid position on its interface display.

[0222] In one embodiment, the remote instruction method and system are faster, more intuitive, and less ambiguous than verbally describing the area of ​​interest.

[0223] In one embodiment, Method 10 may be adapted to further include steps of Method 200 so that a user of a second computer machine can remotely point to an area of ​​interest in the simulator using the same video conferencing platform.

[0224] Similarly, system 100 may be configured to allow a user of a second computer machine 130 to point to an area of ​​interest within the simulator 102. In this case, the first computer 120 is configured to transmit an image of at least a portion of the simulator 102 via a first video conferencing client over a secure public network 138, and the second computer machine 130 is configured to receive an image of the portion of the simulator 102 via a second video conferencing client and to display the received image on a display unit 132. The second computer machine is further configured to capture the coordinates of at least one point selected in the image by a user of the second computer machine 130 and transmit the captured coordinates to the first computer machine 120 via a second video conferencing client over the secure public network 138. Upon receiving coordinates, the first computer machine 120 identifies the corresponding region of interest within the simulator 102 based on the received coordinates in order to visually distinguish the region of interest, and controls a light source present in the simulator 102, such as a projector, to illuminate the region of interest within the simulator 102.

[0225] The embodiments of the invention described above are intended to be illustrative only. Therefore, the scope of the invention is intended to be limited solely by the appended claims.

Claims

1. A computer-based method for remotely pointing to a region of interest of a simulator, wherein the process is: The steps include receiving the coordinates of at least one point selected within a given image of at least a portion of the simulator, wherein the given image is displayed on a display unit, and A step of identifying a region of interest in the simulator based on the received coordinates of the at least one point; The steps include providing a control signal to a projector and projecting light toward the region of interest within the simulator so that a person inside the simulator can visually distinguish the region of interest, A method performed by a computer, including the above.

2. The step of identifying the region of interest within the simulator includes the step of mapping the received coordinates to a pointer area in a coordinate system associated with the projector, The step of providing control signals includes the steps of: generating an overlay image based on the pointer area, wherein the overlay image encompasses the region of interest; and providing the control signals to the projector to project the overlay image on the simulator. The method carried out by a computer as described in claim 1.

3. The step of mapping the received coordinates to the pointer area is: The steps include mapping the received coordinates to at least one given point in the coordinate system of the simulator, The steps include mapping the at least one given point to the pointer area in the coordinate system associated with the projector, A computer-based method according to claim 2, including the method described in claim 2.

4. The steps include establishing a connection between a first video conferencing client running on a first computer and a second video conferencing client running on a second computer over a telecommunications network, The steps include transmitting the given image to the second computer through the first video conferencing client, wherein the second computer provides the given image for display on a remote display unit, The step of receiving the coordinates includes the step of receiving the coordinates from the second computer via the first video conferencing client over the telecommunications network. A computer-based method according to any one of claims 1 to 3.

5. The method implemented by a computer according to claim 4, wherein the telecommunications network includes one of a public network, a secure public network, a private network, and a secure private network.

6. The method performed by a computer according to any one of claims 1 to 5, wherein the step of providing the control signal includes the step of selecting the projector from among a plurality of available projectors based on the received coordinates, and the step of providing the control signal to the selected projector.

7. A computer-based method according to any one of claims 1 to 5, wherein the step of providing the control signal includes the steps of determining a target position for the projector, the target position ensuring that the region of interest is included within the field of view of the projector, and further providing the control signal indicating the target position.

8. A computer-based method according to any one of claims 1 to 7, further comprising the step of receiving a given image of at least a portion of the simulator from a camera located within the simulator.

9. A computer-based method according to any one of claims 1 to 7, further comprising the step of generating the given image of at least a portion of the simulator.

10. The computer-based method according to claim 9, wherein the step of generating the given image is performed using a 3D model of the simulator.

11. A computer-based method according to any one of claims 1 to 10, wherein the step of receiving coordinates includes receiving coordinates for a plurality of points selected in the given image, the step of identifying a region of interest includes identifying a plurality of regions of interest based on the received coordinates for the plurality of points, and the step of providing a control signal includes providing the control signal to the projector so that the person in the simulator can visually and simultaneously distinguish the plurality of regions of interest.

12. The computer-based method according to claim 2, further comprising the step of selecting at least one characteristic of the pointer area, wherein the control signal represents the at least one characteristic.

13. The computer-assisted method according to claim 12, wherein the at least one characteristic includes at least one of shape, strength, and color.

14. A system that remotely points to a region of interest within a simulator, Processor and A non-temporary storage medium operably connected to the processor, comprising a non-temporary storage medium containing computer-readable instructions, When the processor executes the computer-readable instruction, The simulator receives the coordinates of at least one point selected within a given image of at least a portion thereof, and the given image is displayed on a display unit. Based on the received coordinates of the at least one point, the region of interest in the simulator is identified. A control signal is provided to the projector, and light is projected toward the area of ​​interest within the simulator, so that a person inside the simulator can visually distinguish the area of ​​interest. A system configured for that purpose.

15. The aforementioned processor, The received coordinates are mapped to a pointer area in the coordinate system associated with the projector. An overlay image is generated based on the aforementioned pointer area, and the overlay image encompasses the region of interest. The control signal is provided to the projector so that the overlay image is projected onto the simulator. The system according to claim 14, configured for the purpose of

16. The aforementioned processor, The received coordinates are mapped to at least one given point in the coordinate system of the simulator. Mapping the at least one given point to the pointer area in the coordinate system associated with the projector, The system according to claim 15, configured for the purpose of

17. The aforementioned processor, A connection is established between a first video conferencing client running on a first computer and a second video conferencing client running on a second computer over a telecommunications network. The given image is transmitted to the second computer through the first video conferencing client, and the second computer is configured to provide the given image for display on a remote display unit. Receiving the aforementioned coordinates includes receiving the aforementioned coordinates from the second computer via the first video conferencing client over the telecommunications network. The system according to any one of claims 14 to 16.

18. The system according to claim 17, wherein the telecommunications network includes one of a public network, a secure public network, a private network, and a secure private network.

19. The system according to any one of claims 14 to 18, wherein the processor is further configured to select a projector from among a plurality of available projectors based on the received coordinates and to provide the control signal to the selected projector.

20. The system according to any one of claims 14 to 18, wherein the processor is further configured to determine a target position for the projector, the target position ensuring that the region of interest is included within the field of view of the projector, and to provide the control signal further indicating the target position.

21. The system according to any one of claims 14 to 20, wherein the processor is further configured to receive a given image of at least a portion of the simulator from a camera located within the simulator.

22. The system according to any one of claims 14 to 20, wherein the processor is further configured to generate the given image of at least a portion of the simulator.

23. The system according to claim 22, wherein the processor is configured to generate the given image using the 3D model of the simulator.

24. The system according to any one of claims 14 to 23, wherein the processor is configured to receive coordinates for a plurality of points selected in the given image, to identify a plurality of regions of interest based on the received coordinates for the plurality of points, and to provide the control signal to the projector so that the person in the simulator can visually and simultaneously distinguish the plurality of regions of interest.

25. The system according to claim 15, wherein the processor is further configured to select at least one characteristic of the pointer area, and the control signal represents the at least one characteristic.

26. The system according to claim 25, wherein the at least one characteristic includes at least one of shape, strength, and color.

27. A computer program product that remotely points to a region of interest within a simulator, the computer program product includes computer-readable memory storing computer executable instructions, and the computer executable instructions, when executed by a processor, The steps include receiving the coordinates of at least one point selected within a given image of at least a portion of the simulator, wherein the given image is displayed on a display unit, and A step of identifying a region of interest in the simulator based on the received coordinates of the at least one point; The steps include providing a control signal to a projector and projecting light toward the region of interest within the simulator so that a person inside the simulator can visually distinguish the region of interest, A computer program product that performs the steps of the method.

Images