Information processing system and information processing method for vehicle

Abstract

The present application discloses an information processing system and an information processing method for a vehicle, in which the information processing system includes: an image generation unit configured to generate an image based on an input instruction; a communication unit configured to receive and transmit the image generated by the image generation unit and image-related information based on an input instruction; a projection unit coupled with the communication unit and configured to project an image based on the image-related information; and a telescopic display unit configured to extend or retract based on the control of the control switch, in which the telescopic display unit extends and displays the image projected by the projection unit when the control switch is turned on, and the telescopic display unit retracts and does not display the image when the control switch is turned off.

Description

Technical Field

[0001] This disclosure relates to the field of aviation, and more specifically to information processing systems and methods for use in vehicles. Background Technology

[0002] Currently, Head-Up Display (HUD) systems have experienced rapid development and widespread application. In vehicles, HUD systems (also known as head-up display systems) can enhance information display. HUD technology projects important information such as vehicle information, turn instructions, lane departure warnings, navigation, road conditions, and telephone information directly into the field of vision of the vehicle operator (e.g., driver or pilot). This allows the operator to obtain information about the vehicle's movement without looking down or turning their head, simply by looking straight ahead. This maximizes the operator's focus on flight conditions and effectively improves safety.

[0003] Existing HUD technology typically uses windshield reflection for imaging. However, due to the curvature of the windshield surface, the image entering the eye behind the windshield is distorted, causing the image seen by the driver to deviate from the original design and affecting the user experience of the HUD.

[0004] In the aviation field, air traffic control involves ground-based air traffic controllers coordinating and guiding the flight paths and modes of different aircraft within airspace or airports to prevent accidents on the ground or in the air and to ensure smooth operation for maximum efficiency. Typically, communication between air traffic control (ATC) systems or operations control centers (OCCs) and aircraft is based on radio communication via voice calls. This method of communication suffers from being unintuitive and lacking interactivity.

[0005] Therefore, it is desirable to provide an interactive, enhanced display information processing system and information processing method. Summary of the Invention

[0006] A brief overview of this disclosure is given below to provide a basic understanding of certain aspects of it. However, it should be understood that this overview is not an exhaustive summary of this disclosure. It is not intended to identify key or essential parts of this disclosure, nor is it intended to limit the scope of this disclosure. Its purpose is merely to present certain concepts of this disclosure in a simplified form as a prelude to the more detailed description that follows.

[0007] In view of the above problems, the purpose of this disclosure is to provide an interactive, enhanced display information processing system, information processing method and computer-readable storage medium, as well as an aircraft configured with or coupled to the above information processing system.

[0008] According to one aspect of this disclosure, an information processing system for a vehicle is provided, comprising: an image generation unit configured to generate an image based on an input command; a communication unit configured to receive and transmit the image generated by the image generation unit and image-related information based on the input command; a projection unit coupled to the communication unit and configured to project the image based on the image-related information; and a retractable display unit configured to extend or retract based on the control of a control switch, wherein when the control switch is turned on, the retractable display unit extends and displays the image projected by the projection unit, and when the control switch is turned off, the retractable display unit retracts and does not display an image.

[0009] According to embodiments of the present disclosure, the information processing system further includes a monitoring device comprising: one or more sensors configured to monitor at least one of characteristic information of the operator of the vehicle and environmental information of the vehicle; an event determination unit configured to determine whether a predetermined event has occurred based on at least one of the information monitored by the one or more sensors and the driving mode of the vehicle received by the monitoring device; and an event notification unit configured to generate a notification indicating the occurrence of the predetermined event when the event determination unit determines that the predetermined event has occurred, and to transmit the notification to an image generation unit for visual display.

[0010] According to embodiments of this disclosure, the operator's characteristic information includes at least one of the operator's posture, voice, and eye movements.

[0011] According to embodiments of this disclosure, one or more sensors monitor the operator's eye movements.

[0012] According to an embodiment of the present disclosure, the event determination unit determines whether an eye deviation event has occurred based on information about the operator's eye movements monitored by one or more sensors; wherein, when the event determination unit determines that the operator's eye has deviated from the eye box area of ​​the head-up display device arranged on the vehicle, an eye deviation event is determined to have occurred; and wherein, when the event determination unit determines that an eye deviation event has occurred, the event notification unit generates a notification indicating that an eye deviation event has occurred.

[0013] According to embodiments of this disclosure, the image-related information includes at least one of priority information and event notification information.

[0014] According to embodiments of this disclosure, either priority information or the urgency of the event is preset.

[0015] According to embodiments of this disclosure, the communication unit includes at least one of an optical system, a wireless communication system, and a wired communication system.

[0016] According to an embodiment of this disclosure, the retractable display unit is arranged in front of the windshield of a vehicle and is transparent.

[0017] According to an embodiment of this disclosure, the projection unit is configured to project the image differentially based on either priority information or the urgency of the event.

[0018] According to embodiments of this disclosure, the image generation unit is further configured to generate an image based on at least one of an input instruction from an external device connected to the vehicle and an input image.

[0019] According to embodiments of this disclosure, the means of transportation is an aircraft.

[0020] According to one aspect of this disclosure, an information processing method for a vehicle is provided, comprising: generating an image based on an input command; transmitting the generated image and image-related information based on the input command; projecting the image based on the image-related information; and extending or retracting a retractable display unit for display based on the control of a control switch, wherein the retractable display unit extends and displays the projected image when the control switch is turned on, and retracts and does not display the image when the control switch is turned off, wherein the control switch and the retractable display unit are disposed in the vehicle.

[0021] According to embodiments of this disclosure, the information processing method further includes: monitoring at least one of the characteristic information of the operator of the vehicle and the environmental information of the vehicle; determining whether a predetermined event has occurred based on at least one of the monitored information and the driving mode of the vehicle; and, if it is determined that a predetermined event has occurred, generating a notification indicating that the predetermined event has occurred, and transmitting the notification for visual display.

[0022] According to embodiments of this disclosure, the image-related information includes at least one of priority information and event notification information.

[0023] According to embodiments of this disclosure, the event notification information includes at least one of a notification that an event has occurred and the urgency level of the event.

[0024] According to embodiments of this disclosure, either priority information or the urgency of the event is preset.

[0025] According to embodiments of this disclosure, the information processing method further includes generating an image based on at least one of an input instruction from an external device connected to a vehicle and an input image.

[0026] According to one aspect of this disclosure, a computer-readable storage medium is provided, on which a program is stored, the program being used to enable the above-described information processing method of a computer system.

[0027] According to one aspect of this disclosure, an aircraft including the aforementioned information processing system is also provided.

[0028] Other aspects of embodiments of this disclosure are set forth in the following description section, wherein preferred embodiments of the present disclosure are described in detail without limiting them. Attached Figure Description

[0029] To further illustrate the above and other advantages and features of this disclosure, the specific embodiments of this disclosure are described in more detail below with reference to the accompanying drawings. These drawings, together with the following detailed description, are included in and form a part of this specification. Elements having the same function and structure are indicated by the same reference numerals. It should be understood that these drawings only depict typical examples of this disclosure and should not be considered as limiting the scope of this disclosure. In the drawings:

[0030] Figure 1 This is a block diagram illustrating an exemplary structure of an information processing system for a vehicle according to an embodiment of the present disclosure;

[0031] Figure 2 The image below the eye box plane according to an embodiment of the present invention is schematically shown to be mapped to an image below the retinal plane of the pilot.

[0032] Figure 3A A schematic diagram illustrating the application of an information processing system according to an embodiment of the present invention on an aircraft;

[0033] Figure 3B The optical transmission path of an information processing system according to an embodiment of the present invention is illustrated schematically.

[0034] Figure 4 A block diagram illustrating an example functional configuration of a monitoring device for a vehicle according to an embodiment of the present disclosure is shown;

[0035] Figure 5 A flowchart illustrating an example of a monitoring method according to an embodiment of the present disclosure is provided.

[0036] Figure 6 A flowchart illustrating an example of an information processing method according to embodiments of the present disclosure; and

[0037] Figure 7This is a block diagram of an exemplary structure of a general-purpose personal computer in which the methods and / or devices according to embodiments of the present invention can be implemented. Detailed Implementation

[0038] Exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of actual implementations are described in the specification. However, it should be understood that many implementation-specific decisions must be made in the development of any such actual embodiment to achieve the developer’s specific goals, such as complying with constraints related to the system and business, and these constraints may vary from implementation to implementation. Furthermore, it should be understood that while development work can be very complex and time-consuming, such development work is merely a routine task for those skilled in the art who benefit from the present disclosure.

[0039] It should also be noted that, in order to avoid obscuring this disclosure with unnecessary details, only the equipment structure and / or processing steps closely related to the solution according to this disclosure are shown in the accompanying drawings, while other details that are not closely related to this disclosure are omitted.

[0040] The embodiments according to this disclosure are described in detail below with reference to the accompanying drawings.

[0041] First, refer to Figure 1 An implementation example of an information processing system 100 for vehicles according to this disclosure is described. Figure 1 This is a block diagram illustrating an example of the functional configuration of an information processing system 100 for a vehicle according to an embodiment of the present disclosure.

[0042] like Figure 1 As shown, the information processing system 100 according to the embodiments of the present disclosure may include an image generation unit 102, a communication unit 104, a projection unit 106, and a retractable display unit 108.

[0043] The image generation unit 102 can be configured to generate an image based on an input instruction, which can be an instruction sent from the vehicle to the information processing system 100, or an instruction received from an external device connected to the vehicle.

[0044] For example, the means of transportation can be aircraft including passenger planes, cargo planes, helicopters, etc., and the input command can be a command sent by an air traffic controller (ATC) or an operator at the Operations Control Center (OCC). The image generation unit 102 can generate an image such as a taxiway map for pilot navigation based on the received command.

[0045] According to another embodiment of this disclosure, the image generation unit 102 can be configured to generate images based on input commands and input images from an external device connected to the vehicle. For example, the input image may be an image that an ATC controller or OCC operator wishes to share with the vehicle. The image generation unit 102 can then generate an image containing the same information as the input image. In this way, ATC controllers or OCC operators can intuitively share images, such as flight route maps, with the aircraft, thereby improving the efficiency of guiding the flight routes and flight patterns of different aircraft within the airspace or airport.

[0046] The communication unit 104 can be configured to receive and transmit information from various units within the information processing system or external devices. For example, the communication unit 104 can be configured to receive and transmit images generated by the image generation unit 102 and information related to the generated images based on input instructions. The information related to the generated images (i.e., image-related information) may include at least one of priority information and event notification information. The event notification information may include at least one of notification of an event and the urgency level of the event.

[0047] It should be recognized that either priority information or the urgency of an event can be pre-set by pilots, aircraft maintenance personnel, or controllers.

[0048] According to embodiments of the present invention, the communication unit may include at least one of an optical system, a wireless communication system, and a wired communication system. According to one embodiment of the present invention, the communication unit may be an optical system employing a waveguide optics device, commonly referred to as an optical waveguide, for light transmission, wherein the optical waveguide projects electromagnetic waves within the visible portion of the electromagnetic spectrum. It should be understood that the communication unit of the present invention is not limited thereto, and other types of communication units may also be used.

[0049] The projection unit 106 is coupled to the communication unit 104 and can be configured to project an image based on image-related information. According to an embodiment of the present disclosure, the projection unit 106 can map the received image under the eye box plane to an image under the pilot's retinal plane for projection onto the retractable display unit. Finally, the light beam projected onto the retractable display unit is reflected into the pilot's eye.

[0050] Figure 2 This diagram schematically illustrates the mapping between the image below the eye box plane and the image below the pilot's retina plane. Figure 2 The left side schematically shows an image transmitted via communication unit 104 to the eye-box plane of the projection unit. Figure 2 The right side schematically shows the image mapped below the pilot's retinal plane.

[0051] According to an embodiment of the present invention, the projection unit 106 can be configured to perform differentiated projection based on either priority information or the urgency of the event. For example, when there are obstacles around the aircraft, the closer the relative distance between the aircraft and the obstacles, the larger the size of the symbol representing the obstacle (e.g., obstacle icon) in the image projected by the projection unit 106, so as to warn the pilot.

[0052] The retractable display unit 108 can be extended or retracted with a single button. Specifically, the retractable display unit 108 can extend and display the image projected by the projection unit 106 when the control switch is on, and it can retract and not display an image when the control switch is off. According to an embodiment of the present invention, the retractable display unit 108 can be transparent.

[0053] According to embodiments of this disclosure, the information processing system 100 may optionally include a monitoring device 110. The configuration of the monitoring device 110 will be described below and will not be repeated here.

[0054] Figure 3A This schematic diagram illustrates an information processing system according to an embodiment of the present invention applied on an aircraft. The retractable display unit 308 can be positioned near the sun visor between the pilot and the windshield 302. The optical path transmission between the image generation unit 306, the projection unit, and the retractable display unit 308 is as follows: Figure 3A As shown.

[0055] Figure 3B The optical transmission path of an information processing system according to an embodiment of the present invention is illustrated schematically. The information processing system according to an embodiment of the present invention has a small size, making it suitable for deployment in confined spaces.

[0056] Below, we will refer to Figure 4 A monitoring device 400 for a vehicle is described according to an embodiment of the present disclosure. Figure 4 A block diagram illustrating an example functional configuration of a monitoring device for a vehicle according to an embodiment of the present disclosure is shown.

[0057] like Figure 4 As shown, a monitoring device 400 for a vehicle according to an embodiment of the present disclosure may include one or more sensors 402, an event determination unit 404, and an event notification unit 406.

[0058] One or more sensors 402 may be configured to monitor at least one of the operator's characteristic information and the vehicle's environmental information, wherein the operator's characteristic information may include at least one of the operator's posture, voice, and eye movements.

[0059] For example, one or more sensors 402 may be arranged inside or outside the cabin of the aircraft or installed on a wearable device worn by the pilot to monitor at least one of the pilot's posture, voice and eye movements, such as monitoring the pilot's gestures.

[0060] The event determination unit 404 can be configured to determine whether a predetermined event has occurred based on at least one of the information monitored by one or more sensors 402 and the vehicle's driving mode (i.e., the vehicle's operating mode) received by the monitoring device 400.

[0061] According to an embodiment of the present invention, graphic symbols corresponding to different flight modes (e.g., taxiing mode, parking mode, landing mode, etc.) can be displayed on the aircraft's head-up display device.

[0062] The event notification unit 406 can be configured to generate a notification indicating the occurrence of a predetermined event when the event determination unit 404 determines that a predetermined event has occurred, and to transmit the notification for visual display.

[0063] According to an embodiment of the present invention, a near-infrared light sensor arranged in front of the pilot's seat can be configured to track the pilot's eye movements. When the event determination unit 404 determines that the pilot's eyeballs deviate from the eye box area of ​​the head-up display device arranged on the aircraft (e.g., the pilot's line of sight is outside the boundary of the eye box area), the event determination unit 404 determines that an eyeball deviation event has occurred.

[0064] When the event determination unit 404 determines that an eye deviation event has occurred, the event determination unit 404 transmits the event determination result to the event notification unit 406. The event notification unit 406 can generate a notification indicating that an eye deviation event has occurred and transmit the notification.

[0065] It should be recognized that the monitoring device 400 for vehicles according to this disclosure can be implemented as part of the aforementioned information processing system 100 (i.e., as part of...). Figure 1 (The optional monitoring device 110 shown is used for operation).

[0066] In this case, when the information processing system 100 includes an optional monitoring device 110, in the event of a predetermined event, the event notification unit of the monitoring device 110 will transmit a notification indicating that a predetermined event has occurred (e.g., a change in driving mode) to the image generation unit of the information processing system 100 for processing, thereby visually displaying an image for notifying the occurrence of the predetermined event.

[0067] For example, when an event occurs that changes the flight mode of an aircraft, the event notification unit of the monitoring device 110 sends a notification of the event to the information processing system 100 in the form of an instruction. The image generation unit 102, the projection unit 104, and the retractable display unit 106 in the information processing system 100 can process the event as discussed above to visually notify the pilot of the flight mode change. For example, the pilot can be notified of the flight mode change by highlighting the current flight mode of the aircraft or by flashing the symbol indicating the flight mode change on the display unit.

[0068] The monitoring device 400 for vehicles according to this disclosure can also operate independently of the information processing system 100 according to this disclosure. In the event of a predetermined event, the event notification unit 406 of the monitoring device 400 can transmit a notification indicating the occurrence of the predetermined event to a system other than the information processing system 100 of this disclosure for visual display.

[0069] Figure 5 A flowchart of a monitoring method corresponding to the aforementioned monitoring device 400 is shown. For example... Figure 5 As shown, the monitoring method 500 according to the embodiments of this disclosure may begin at step S502 and may include steps S504-S508.

[0070] In step S504, at least one of the characteristic information of the operator of the vehicle and the environmental information of the vehicle is monitored. Step S504 can be implemented by one or more sensors 402 in the monitoring device 400 according to the embodiments of the present disclosure described above. Therefore, for details, please refer to the description above, and it will not be repeated here.

[0071] In step S506, it is determined whether a predetermined event has occurred based on at least one of the monitored information and the vehicle's driving mode. Step S506 can be implemented by the event determination unit 404 in the monitoring device 400 according to the embodiments of the present disclosure described above. Therefore, for details, please refer to the description above, and it will not be repeated here.

[0072] If it is determined in step S506 that a predetermined event has occurred ("Yes" in step S506), the process can proceed to step S508. In step S508, a notification indicating that the predetermined event has occurred is generated and transmitted for visual display. Step S508 can be implemented by the event notification unit 406 in the monitoring device 400 according to the embodiment of this disclosure described above; therefore, specific details can be found in the description above and will not be repeated here.

[0073] On the other hand, if it is determined in step S506 that no predetermined event has occurred (No in step S506), the process can end (step S510).

[0074] Compared with the above reference Figure 1 Corresponding to the information processing system 100 according to the embodiments of the present disclosure described in FIG3, the present disclosure also provides an information processing method for vehicles.

[0075] Figure 6 This is a flowchart illustrating an example of the flow of an information processing method 600 according to an embodiment of the present disclosure. For example... Figure 6 As shown, the information processing method 600 according to the embodiments of the present disclosure may begin at the start step S602 and may include steps S604-S610.

[0076] In step S604, an image is generated based on an input instruction. The input instruction can be an instruction sent by the vehicle to the information processing system, or it can be an instruction received from an external device connected to the vehicle. Step S604 can be implemented by the image generation unit 102 described above. Therefore, for details, please refer to the description above, and it will not be repeated here.

[0077] In step S606, the generated image and image-related information based on the input command are transmitted. The information related to the generated image (i.e., image-related information) may include at least one of priority information and event notification information. The event notification information may include at least one of notification of an event and the urgency level of the event. Step S606 may be implemented by the communication unit 104 described above. Therefore, for specific details, please refer to the description above, and it will not be repeated here.

[0078] In step S608, the received image is projected based on image-related information. Step S608 can be implemented by the projection unit described above. For details, please refer to the description above. It will not be repeated here.

[0079] In step S610, the retractable display unit extends or retracts based on the control of a control switch for display. When the control switch is open, the retractable display unit extends and displays the projected image; when the control switch is closed, the retractable display unit retracts and does not display an image. The control switch and the retractable display unit are located in the vehicle. The retractable display unit 108 can be a one-button open type. According to an embodiment of the present invention, the retractable display unit 108 can be transparent. Step S610 can be implemented by the retractable display unit 108 described above; specific details can be found in the above description and will not be repeated here.

[0080] Then, the information processing method 600 may end at step S612.

[0081] Optionally, according to the information processing method 600 of the present disclosure, before performing step S604, the monitoring processing described in the monitoring method 500 above can be performed, and then the monitored events (e.g., changes in the flight mode of the aircraft) are visualized in steps S604-S610.

[0082] The methods discussed above can be implemented entirely by computer-executable programs, or partially or entirely using hardware and / or firmware.

[0083] Each component module and unit in the aforementioned device or system can be configured via software, firmware, hardware, or a combination thereof. Specific means or methods of configuration are well known to those skilled in the art and will not be elaborated upon here. When implemented via software or firmware, data can be transferred from a storage medium or network to a computer with a dedicated hardware architecture (e.g., [missing information]). Figure 7 The general-purpose computer 700 shown is equipped with the programs that constitute the software, and when various programs are installed, the computer is able to perform various functions, etc.

[0084] Figure 7 This is a block diagram of an exemplary structure of a general-purpose personal computer in which methods and / or networks according to embodiments of the present invention can be implemented.

[0085] like Figure 7 As shown, the Central Processing Unit (CPU) 701 performs various processes based on programs stored in the Read-Only Memory (ROM) 702 or programs loaded into the Random Access Memory (RAM) 703 from the storage device 708. The RAM 703 also stores data required as needed when the CPU 701 performs various processes, etc. The CPU 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output interface 705 is also connected to the bus 704.

[0086] The following components are connected to the input / output interface 705: input device 706 (including keyboard, mouse, etc.), output device 707 (including display, such as cathode ray tube (CRT), liquid crystal display (LCD), etc., and speaker, etc.), storage device 708 (including hard disk, etc.), and communication device 709 (including network interface card, such as LAN card, modem, etc.). The communication device 709 performs communication processing via a network, such as the Internet.

[0087] If necessary, drive 710 can also be connected to input / output interface 705. Removable media 711, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 710 as needed, so that computer programs read from them can be installed into storage device 708 as needed.

[0088] When the above series of processes are implemented through software, the program constituting the software is installed from a network such as the Internet or a storage medium such as removable media 711.

[0089] Those skilled in the art will understand that such storage media are not limited to Figure 7 The removable medium 711 shown contains programs and is distributed separately from the device to provide programs to users.

[0090] Examples of removable media 711 include disks (including floppy disks (registered trademark)), optical disks (including optical disc read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including mini-disk (MD) (registered trademark)) and semiconductor memory. Alternatively, the storage medium may be ROM 702, a hard disk included in storage device 708, etc., containing programs and distributed to the user along with the device containing them.

[0091] The present invention also provides corresponding computer program code and a computer program product storing machine-readable instruction code. When the instruction code is read and executed by a machine, the method described above according to the embodiments of the present invention can be performed.

[0092] Accordingly, storage media configured to carry the aforementioned program product storing machine-readable instruction code are also included in the disclosure of this invention. These storage media include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, etc.

[0093] Preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, but the present disclosure is by no means limited to the examples described above. Various changes and modifications can be made by those skilled in the art within the scope of the appended claims, and it should be understood that such changes and modifications naturally fall within the technical scope of the present disclosure.

[0094] For example, the multiple functions included in one unit in the above embodiments can be implemented by separate devices. Alternatively, the multiple functions implemented by multiple units in the above embodiments can be implemented by separate devices respectively. In addition, one of the above functions can be implemented by multiple units. Needless to say, such a configuration is included within the scope of the present disclosure.

[0095] In this specification, the steps described in the flowchart include not only processes executed sequentially in the stated order, but also processes executed in parallel or individually, rather than necessarily sequentially. Furthermore, even within the steps of sequential processing, needless to say, the order can be appropriately altered.

[0096] Based on the above description, the embodiments of this disclosure provide the following technical solutions, but are not limited thereto.

[0097] Option 1. An information processing system for vehicles, comprising:

[0098] An image generation unit configured to generate an image based on an input instruction;

[0099] A communication unit is configured to receive and transmit images generated by an image generation unit and image-related information based on input instructions;

[0100] A projection unit, which is coupled to a communication unit and configured to project an image based on image-related information;

[0101] The retractable display unit is configured to extend or retract based on the control of a control switch, wherein when the control switch is turned on, the retractable display unit extends and displays an image projected by the projection unit, and when the control switch is turned off, the retractable display unit retracts and does not display an image.

[0102] Option 2. The information processing system according to Option 1 further includes a monitoring device, which includes:

[0103] One or more sensors, wherein the one or more sensors are configured to monitor at least one of the characteristic information of the operator of the vehicle and the environmental information of the vehicle;

[0104] An event determination unit is configured to determine whether a predetermined event has occurred based on at least one of information monitored by one or more sensors and the vehicle's driving mode received by a monitoring device; and

[0105] The event notification unit is configured to generate a notification indicating the occurrence of a predetermined event when the event determination unit determines that a predetermined event has occurred, and to transmit the notification to the image generation unit for visual display.

[0106] Option 3. According to the information processing system described in Option 2, the operator's characteristic information includes at least one of the operator's posture, voice, and eye movements.

[0107] Option 4. According to the information processing system described in Option 3, one or more sensors monitor the operator's eye movements.

[0108] Option 5. The information processing system described in Option 4,

[0109] The event determination unit determines whether an eye deviation event has occurred based on information about the operator's eye movements monitored by one or more sensors.

[0110] Specifically, when the event determination unit determines that the operator's eyeball has deviated from the eye box area of ​​the head-up display device located on the vehicle, it determines that an eyeball deviation event has occurred; and

[0111] Specifically, when the event determination unit determines that an eye deviation event has occurred, the event notification unit generates a notification indicating that an eye deviation event has occurred.

[0112] Option 6. The information processing system according to Option 1, wherein image-related information includes at least one of priority information and event notification information.

[0113] Option 7. The information processing system according to Option 6, wherein the event notification information includes at least one of a notification that an event has occurred and the urgency level of the event.

[0114] Option 8. The information processing system according to Option 7, wherein either priority information or the urgency of the event is preset.

[0115] Option 9. The information processing system according to Option 1, wherein the communication unit includes at least one of an optical system, a wireless communication system, and a wired communication system.

[0116] Option 10. The information processing system according to Option 1, wherein the retractable display unit is arranged in front of the windshield of the vehicle and is transparent.

[0117] Option 11. The information processing system according to Option 8, wherein the projection unit is configured to perform differentiated projection of the image based on either priority information or the urgency of the event.

[0118] Option 12. The information processing system according to Option 1, wherein the image generation unit is further configured to generate an image based on at least one of an input instruction from an external device connected to the vehicle and an input image.

[0119] Option 13. According to any one of Options 1 to 12, the means of transportation is an aircraft.

[0120] Option 14. An information processing method for vehicles, comprising:

[0121] Generate images based on input commands;

[0122] Transmit the generated image and image-related information based on the input command;

[0123] Projecting images based on image-related information;

[0124] A retractable display unit extends or retracts for display based on the control of a control switch, wherein when the control switch is turned on, the retractable display unit extends and displays the projected image, and when the control switch is turned off, the retractable display unit retracts and does not display an image, wherein the control switch and the retractable display unit are installed in a vehicle.

[0125] Option 15. The information processing method according to Option 14 further includes:

[0126] Monitoring at least one of the operator's characteristics and the vehicle's environmental information;

[0127] Determine whether a predetermined event has occurred based on at least one of the monitored information and the vehicle's driving pattern; and

[0128] If a predetermined event is determined to have occurred, a notification indicating that the predetermined event has occurred is generated and transmitted for visual display.

[0129] Solution 16. The information processing method according to Solution 14, wherein the image-related information includes at least one of priority information and event notification information.

[0130] Option 17. The information processing method according to Option 16, wherein the event notification information includes at least one of a notification that an event has occurred and the urgency level of the event.

[0131] Option 18. The information processing method according to Option 17, wherein either priority information or the urgency of the event is preset.

[0132] Solution 19. The information processing method according to Solution 14, wherein the information processing method further includes generating an image based on at least one of an input instruction from an external device connected to the vehicle and an input image.

[0133] Scheme 20. A computer-readable storage medium having a program stored thereon for causing a computer system to perform the information processing method described in any one of Schemes 14 to 19.

[0134] Option 21. A method for monitoring vehicles, comprising:

[0135] Monitoring at least one of the operator's characteristics and the vehicle's environmental information;

[0136] Determine whether a predetermined event has occurred based on at least one of the monitored information and the vehicle's driving pattern; and

[0137] If a predetermined event is determined to have occurred, a notification indicating that the predetermined event has occurred is generated and transmitted for visual display.

[0138] Option 22. According to the monitoring method described in Option 21, the operator's characteristic information includes at least one of the operator's posture, voice, and eye movements.

[0139] Option 23. A monitoring device for vehicles, comprising:

[0140] One or more sensors, wherein the one or more sensors are configured to monitor at least one of the characteristic information of the operator of the vehicle and the environmental information of the vehicle;

[0141] An event determination unit is configured to determine whether a predetermined event has occurred based on information monitored by one or more sensors and at least one of the vehicle's driving mode; and

[0142] The event notification unit is configured to generate a notification indicating the occurrence of a predetermined event when the event determination unit determines that a predetermined event has occurred, and to transmit the notification to the image generation unit for visual display.

[0143] Option 24. According to the monitoring device of Option 23, the operator's characteristic information includes at least one of the operator's posture, voice, and eye movements.

[0144] Option 25. According to the monitoring device of Option 24, one or more sensors monitor the operator's eye movements.

[0145] Option 26. The monitoring device according to Option 25,

[0146] The event determination unit determines whether an eye deviation event has occurred based on information about the operator's eye movements monitored by one or more sensors.

[0147] Specifically, when the event determination unit determines that the operator's eyeball has deviated from the eye box area of ​​the head-up display device located on the vehicle, it determines that an eyeball deviation event has occurred; and

[0148] Specifically, when the event determination unit determines that an eye deviation event has occurred, the event notification unit generates a notification indicating that an eye deviation event has occurred.

[0149] Option 27. The monitoring device according to any one of Options 23 to 26, wherein the means of transportation is an aircraft.

[0150] Finally, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Furthermore, unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0151] While embodiments of the present invention have been described in detail above with reference to the accompanying drawings, it should be understood that the embodiments described above are merely illustrative of the invention and do not constitute a limitation thereof. Those skilled in the art can make various modifications and alterations to the above embodiments without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined only by the appended claims and their equivalents.

Claims

1. An information processing system for vehicles, comprising: An image generation unit configured to generate an image based on an input instruction; A communication unit is configured to receive and transmit images generated by the image generation unit and image-related information based on the input command. A projection unit, coupled to the communication unit and configured to project the image based on the image-related information; A retractable display unit is configured to extend or retract based on the control of a control switch, wherein when the control switch is turned on, the retractable display unit extends and displays an image projected by the projection unit, and when the control switch is turned off, the retractable display unit retracts and does not display an image.

2. The information processing system according to claim 1 further includes a monitoring device, the monitoring device comprising: One or more sensors, the one or more sensors being configured to monitor at least one of characteristic information of the operator of the vehicle and environmental information of the vehicle; An event determination unit is configured to determine whether a predetermined event has occurred based on at least one of the information monitored by the one or more sensors and the driving mode of the vehicle received by the monitoring device. as well as An event notification unit is configured to generate a notification indicating the occurrence of the predetermined event when the event determination unit determines that the predetermined event has occurred, and to transmit the notification to the image generation unit for visual display.

3. The information processing system according to claim 2, wherein the operator's characteristic information includes at least one of the operator's posture, voice, and eye movements.

4. The information processing system according to claim 3, wherein the one or more sensors monitor the operator's eye movements.

5. The information processing system according to claim 4, in, The event determination unit determines whether an eye deviation event has occurred based on information about the operator's eye movements monitored by the one or more sensors. Wherein, when the event determination unit determines that the operator's eyeball has deviated from the eye box area of ​​the head-up display device arranged on the vehicle, it determines that the eyeball deviation event has occurred; and When the event determination unit determines that the eyeball deviation event has occurred, the event notification unit generates a notification indicating that the eyeball deviation event has occurred.

6. The information processing system according to claim 1, wherein the image-related information includes at least one of priority information and event notification information.

7. The information processing system according to claim 6, wherein the event notification information includes at least one of a notification that an event has occurred and the urgency level of the event.

8. The information processing system according to claim 7, wherein either the priority information or the urgency of the event is preset.

9. The information processing system according to claim 1, wherein the communication unit comprises at least one of an optical system, a wireless communication system, and a wired communication system.

10. The information processing system of claim 1, wherein the retractable display unit is arranged in front of the windshield of the vehicle and is transparent.

11. The information processing system according to claim 8, wherein, The projection unit is configured to project the image differently based on either the priority information or the urgency of the event.

12. The information processing system of claim 1, wherein the image generation unit is further configured to generate the image based on at least one of an input instruction from an external device connected to the vehicle and an input image.

13. The information processing system according to any one of claims 1 to 12, wherein the means of transportation is an aircraft.

14. An information processing method for a means of transportation, comprising: Generate images based on input commands; Transmit the generated image and image-related information based on the input command; Project the image based on the image-related information; A retractable display unit extends or retracts for display based on a control switch, wherein when the control switch is turned on, the retractable display unit extends and displays the projected image, and when the control switch is turned off, the retractable display unit retracts and does not display an image, wherein the control switch and the retractable display unit are disposed in the vehicle.

15. The information processing method according to claim 14, further comprising: Monitoring at least one of the operator's characteristic information and the environmental information of the vehicle; Determine whether a predetermined event has occurred based on at least one of the monitored information and the vehicle's driving mode; as well as If the predetermined event is determined to have occurred, a notification indicating the occurrence of the predetermined event is generated, and the notification is transmitted for visual display.

16. The information processing method according to claim 14, wherein the image-related information includes at least one of priority information and event notification information.

17. The information processing method according to claim 16, wherein the event notification information includes at least one of a notification that an event has occurred and the urgency level of the event.

18. The information processing method according to claim 17, wherein either the priority information or the urgency of the event is preset.

19. The information processing method of claim 14, wherein the information processing method further comprises generating the image based on at least one of an input instruction from an external device connected to the vehicle and an input image.

20. A computer-readable storage medium having a program stored thereon, the program being used to cause a computer system to perform the information processing method of any one of claims 14 to 19.

Images