DISPLAY SYSTEM ON A SIDE WINDOW ON THE VEHICLE

Abstract

A display system (100) for a side window on vehicles (10) is provided. The display system (100) for a side window on vehicles (10) has a transparent display (170), at least one external sensor (150), at least one internal sensor (160), and a controller (110). The transparent display (170) is provided on a side window of a vehicle (10). The external sensor (150) is used to detect the external environment of the vehicle (10) in order to provide information about the external environment. The internal sensor (160) is used to detect an internal environment of the vehicle (10) in order to provide information about the internal environment.The controller (110) is coupled to the transparent display (170), the external sensor (150), and the internal sensor (160) to select an operation mode based on the information about the internal environment, and controls the transparency distribution and the brightness distribution of the transparent display (170) based on the operation mode and the information about the external environment.

Description

BACKGROUNDFIELD OF DISCLOSUREThe present disclosure relates to a display system, and more particularly, to a display system for a side window of vehicles.Description of Related ArtAs a vehicle continues to move, the scene also changes continuously in front of the window. The side windows of a cockpit of a car are usually made of transparent glass in order to block interference from the outside. Apart from this, it seems difficult to use the flat space of the side windows of the cockpit of the car in another way, for example by combining it with the external environment to generate interaction. Moreover, safety warnings of vehicles are usually displayed on the instrument panel by sounds, lights, or warning messages, but the warning effect is not good for vehicle occupants other than the driver. Thus, in view of the foregoing, if it is possible to display information through the side window, the applicability and safety of the vehicle could be effectively improved.SUMMARY OF THE DISCLOSUREThe present disclosure provides a side window display system for vehicles. By arranging a transparent display on the side window of the vehicle and by arranging a sensor inside and outside the vehicle to provide environmental information inside and outside the vehicle, it is possible to enable passengers to have a virtual and real interactive experience with the image outside the window, thereby improving immersive experience and applicability of vehicles. Moreover, the warning content may be displayed on the transparent display of the car window to achieve a more intuitive warning effect and improve the safety of the vehicle.In the disclosure, a display system for a side window on vehicles includes a transparent display, at least one external sensor, at least one internal sensor, and a controller. The transparent display is disposed on a side window of a vehicle. The external sensor is arranged to sense an external environment of the vehicle to provide information about the external environment. The internal sensor is arranged to sense an internal environment of the vehicle to provide information about the internal environment. The controller is coupled to the transparent display, the external sensor, and the internal sensor to select an operation mode based on the information about the internal environment, and controls the transparency distribution and the brightness distribution of the transparent display based on the operation mode and the information about the external environment.Based on the above, in the side glass display system of the embodiment of the present disclosure, the controller may acquire the current environment information inside and outside the vehicle through the sensors disposed inside and outside the vehicle to control the transparent display to perform transparency and brightness adjustment accordingly. In this way, it is possible to create an environment for passengers in which they have a virtual and real interaction with the images outside the pane, thereby improving the immersive experience and improving the applicability of the vehicle.In order to make the above-mentioned features and advantages of the present disclosure more apparent and easily understood, embodiments will be given below and described in detail with reference to the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1A is a schematic configuration diagram of a display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 1B is a schematic system diagram of a display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 2 is a flowchart of a brightness setting mode of a display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 3 is a flowchart of a parking warning mode of the side window display system on vehicles according to an embodiment of the present disclosure. FIG. 4 is a schematic diagram showing an operation diagram of a display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 5A is a schematic diagram showing the process flow of an AR navigation algorithm of a side window display system on vehicles according to an embodiment of the present disclosure. FIG. 5B is a flowchart of the AR navigation mode of the display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 5C is a schematic diagram illustrating the AR navigation mode of the display system for a side window on vehicles according to an embodiment of the present disclosure. FIGS. 5D to 5G are schematic diagrams illustrating the operation in the AR navigation mode of the display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 5H is a flowchart of an online processing process of the AR navigation algorithm of the side window display system on vehicles according to an embodiment of the present disclosure. FIG. 6A is a schematic diagram showing the flow of a game algorithm of a side window display system on vehicles according to an embodiment of the present disclosure. FIG. 6B is a flowchart of a game mode of a display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 6C is a schematic diagram illustrating operation of the side window display system on vehicles in the game mode according to an embodiment of the present disclosure. FIG. 7 is a flowchart of an electronic whiteboard mode of the display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 8 is a flowchart of a video conferencing mode of the side window display system on vehicles according to an embodiment of the present disclosure. FIG. 9 is a flow chart illustrating conference records in video conferencing mode of the side window display system on vehicles according to an embodiment of the present disclosure. FIG. 10 is a flowchart showing the operation of triggering the display to turn off in the display system for a side window of vehicles according to an embodiment of the present disclosure. FIG. 11 is a flowchart showing the operation of triggering the passenger sleep mode of the side window display system on vehicles according to an embodiment of the present disclosure.DESCRIPTION OF THE EMBODIMENTSUnless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art. It will be further understood that terms, as defined in commonly used dictionaries, should be construed as having a meaning that is consistent with their meaning in the context of the relevant art and disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.It should be understood that the terms "first", "second", and "third" may be used to describe various elements, components, regions, layers, and / or parts in the disclosure, but these elements, components, regions, layers, and / or parts should not be limited by these terms. These terms are used only to distinguish the elements, components, regions, layers and / or parts from one another. Therefore, "first elements", "components", "regions", "layers" and / or "portions" may also be referred to as second elements, components, regions, layers and / or portions without deviating from the scope of application of the inventive concept.The term used herein is used to describe certain embodiments and is not particularly limited. The articles "a", "an", and "the" are intended to include the plural forms as well as "at least one" unless the context clearly indicates otherwise. Unless otherwise indicated, "or" means "and / or.". As used herein, the term "and / or" includes any single element and any combination of two or more of the listed elements. It is also to be understood that the terms "comprises," "includes," and "has" indicate the presence of certain features, regions, steps, operations, elements, components, and / or combinations thereof, but the presence or addition of one or more other features, regions, steps, operations, elements, components, and / or combinations thereof is not excluded.FIG. 1A is a schematic configuration diagram of a display system for a side window on vehicles according to an embodiment of the present disclosure. FIG. 1B is a schematic system diagram of a display system for a side window on vehicles according to an embodiment of the present disclosure. See also FIGS. 1A and 1B. In this embodiment, the side glass display system 100 may be mounted on (or disposed in) the vehicle 10, and the side glass display system 100 includes, for example, a cockpit domain controller (CDC) 110, an electronic compass 120, a positioning system 130, a wireless communication module 140, at least one external sensor 150, at least one internal sensor 160, a transparent display 170, a door interior LED (LED=light emitting diode) 180, an external display 190, and a speaker 191. The cockpit domain controller 110 is coupled to the electronic compass 120, the positioning system 130, the wireless communication module 140, at least one external sensor 150 and at least one internal sensor 160, the transparent display 170, the LED 180 at the door interior, the external display 190 and the loudspeaker 191.The transparent display 170 may be disposed on a side window of a vehicle, for example, may be mounted on the glass on the inside of the side window of the vehicle or integrated into the side window of the vehicle so that passengers in the vehicle 10 may see the transparent display 170. At least one external sensor 150 is arranged to sense the external environment of the vehicle 10 to provide information about the external environment (e.g., light brightness and / or images of the external environment). At least one internal sensor 160 is arranged to sense the internal environment of the vehicle 10 to provide information about the internal environment (e.g., internal environment light brightness, passenger voice, and / or passenger images). The cockpit domain controller 110 selects an operation mode based on the information on the internal environment, and controls the transparency distribution (i.e., the transparency of each pixel in the transparent display 170) and the brightness distribution (i.e., the brightness of each pixel in the transparent display 170) of the transparent display 170 based on the operation mode and the information on the external environment.By arranging sensors in the above sense, inside and outside the vehicle, the cockpit domain controller 110 may capture the current environmental information inside and outside the vehicle to control the transparent display 170 to make appropriate transparency and brightness adjustment, thereby offering passengers an environment in which they may have virtual and real interactions with the images outside the pane, thereby enhancing the immersive experience and improving the applicability of the vehicle.In an embodiment of the present disclosure, the cockpit domain controller 110 may be implemented by a computer on wheels, an onboard computer, or an electronic control unit (ECU=Electronic Control Unit). The positioning system 130 may be implemented by a global positioning system (GPS). The wireless communication module 140 may be implemented by vehicle-to-everything (V2X) third generation mobile phone mobile communication technology standards (3G), fourth generation mobile phone mobile communication technology standards (4G), 4.5G, and / or fifth generation mobile phone mobile communication technology standards (5G), but the embodiment of the present disclosure is not limited thereto.At least one external sensor 150 may include, but is not limited to, an external camera 151 and an external light sensor 152. At least one internal sensor 160 may include, but is not limited to, an internal camera 161 (e.g., an infrared camera), an internal light sensor 162, a microphone 163, and a touch panel 164.In an embodiment of the present disclosure, the transparent display 170 is, for example, a micro light emitting diode display, but the embodiment of the present disclosure is not limited thereto.In an embodiment of the present disclosure, the internal camera 161 can recognize the eye angle of the eyes and the posture of the passenger. Moreover, the internal camera 161 may recognize, via the passenger monitor system (PMS), whether persons are present on the seat, the head angle, the facial features, the opening / closing of the eyes, and the opening / closing of the mouth, and may determine whether the passenger is going to slap, look aside, etc. Through gaze tracking, the internal camera 161 may recognize changes in the eyeballs and the head angle of the passenger, thus identifying a feature of interest (TOI= Token of Interest) and determine the intention of the passenger to leave the vehicle. The internal camera 161 may detect the body movements of the passenger by gesture recognition, thereby controlling the screen display and determining the intention of the passenger to leave the vehicle.In an embodiment of the present disclosure, the external camera 151 may capture the images of the external environment outside the vehicle 10. Moreover, the external camera 151 can recognize objects in the external environment by object recognition, thereby creating a list of possible TOI and determining the distance to the approaching vehicle behind the door opening side. By edge recognition, the external camera 151 can recognize the contour of objects in the external environment to achieve the purpose of virtual and real integration.In an embodiment of the present disclosure, the microphone 163 may capture a voice signal of the passenger. In addition, the microphone 163 can convert the voice commands of the passenger into text by voice recognition. By speech recognition and text-to-speech functions, the microphone 163 can be used together with the speaker 191 to implement a speech assist function.In an embodiment of the present disclosure, the touch panel 164 may detect a touch or a pressure of the passenger on the transparent display 170.FIG. 2 is a flowchart of a brightness setting mode of a display system for a side window on vehicles according to an embodiment of the present disclosure. In this regard, see FIGS. 1A, 1B, and 2. in this embodiment, the brightness adjustment mode can be executed permanently, i.e., at any time and everywhere. In the brightness adjustment mode, the side window display system 100 may use the external light sensor 152 to sense the light brightness of external environment (or external light brightness) outside the vehicle 10 and use the internal light sensor 162 to sense the internal light brightness inside the vehicle 10. Then, the display system 100 for a side glass may perform appropriate computations (such as brightness adjustment and transparency logic) and use to control the transparency distribution and brightness distribution of the transparent display 170 based on the brightness errors of the internal brightness and external brightness. For example, when the brightness inside the vehicle and the brightness outside the vehicle are different, the side glass display system 100 may automatically adjust the brightness and transparency of the transparent display 170.In addition, the brightness adjustment mode may include the following steps. In step S 210, the brightness of the transparent display is X % (X is the initial value), and the transparency of the transparent display is Y % (Y is the initial value). In step S220, it is determined whether |A-B| is larger than T diff where A is the value of the light brightness of the external environment, B is the value of the brightness inside the vehicle 10, and T diff is the threshold value for the difference between internal and external brightness. If |A-B| is not greater than T diff then this means that the difference in brightness between the inside and outside environments is small and does not impair the vision of the passengers. Therefore, the brightness and transparency of the transparent display 170 need not be adjusted (i.e., return to step S 210); conversely, if |A-B| is greater than T diff then this means that the difference in light brightness of the internal and external environments is too large, which could impair the vision of the passenger. Therefore, the brightness and transparency of the transparent display 170 need to be adjusted (i.e., step S 230 is performed).In step S 230, the parameters d 1 and d 2 for adjusting the brightness and transparency of the transparent display 170 may be determined, where d 1= (| A - B|-Tout)*bias_ 1, d 2= (| A - B|-Tout)*bias_ 2. Tout is the brightness threshold of the external environment, d 1 is the dimming brightness ratio of the internal and external brightness differences, bias_ 1 is the dimming weight of the internal and external brightness differences, d 2 is the adjusted transparency ratio of the internal and external brightness differences, and bias_ 2 is the adjusted transparency weight for the internal and external brightness differences.In step S 240, the brightness d 1% and the transparency d 2% of the transparent display 170 may be adjusted. In the embodiment of the present disclosure, the brightness of the transparent display 170 may be increased (e.g., set to X+d 1% ) and the transparency of the display may be decreased (e.g., set to Y-d 2% ) when A>B (i.e., when the external light is brighter); conversely, when A>B (i.e., when the internal light is brighter), the brightness of the transparent display 170 may be decreased (e.g., set to X-d 1% ) and the transparency of the display may be increased (e.g., set to Y+d 2% ).FIG. 3 is a flowchart of a parking warning mode of the side window display system on vehicles according to an embodiment of the present disclosure. In this regard, see FIGS. 1A, 1B and 3. In this embodiment, the parking warning mode can be executed permanently, i.e. at any time and everywhere. In the parking warning mode, the side glass display system 100 may use the internal camera 161 to recognize the gaze and body movements of the passenger, and use the external camera 151 to recognize vehicles coming from behind. Then, when the vehicle is parked, the cockpit domain controller 110 of the side window display system 100 may perform appropriate computations (such as passenger monitoring system (PMS) and object detection) and use to determine whether the passenger should open the door and determine whether a vehicle approaches behind the vehicle, thereby controlling the transparency distribution and brightness distribution of the transparent display 170. That is, when a passenger exits the vehicle and a vehicle approaches from behind, a warning message may be provided to the passenger from an approaching vehicle, and a warning message to warn about the passenger just exiting the vehicle [and about to open] the door may be provided to the vehicle approaching from behind. In addition, as the distance to the rear vehicle becomes narrower, the warning provided may be more noticeable.The following steps may be performed in the brightness adjustment mode. In step S 310, it is determined whether the vehicle speed is zero (i.e., =0), i.e., to determine whether the vehicle has stopped, and the information on the vehicle speed may be provided from the vehicle 10 or calculated by the side glass display system 100 based on the position information provided from the positioning system 130. If the vehicle has not stopped, step S310 is further executed, if the vehicle has stopped, step S320 is executed to determine whether a door opening warning is to be issued. In step S320, it is determined whether the value for the door opening intention of the passenger S total is larger than the door opening threshold value T 1 and whether the distance to the rear vehicle is smaller than the warning threshold value T 2. In an embodiment of the present disclosure, the value of the door opening intention S total may be divided into four parts for evaluation (such as the gaze range of the passenger, the facial angle of the passenger, the rotation angle of the body of the passenger, and the distance between the hand of the passenger and the door handle), and the values of the four parts may be directly added or a weighted sum may be used, which may be determined according to the system design, and the embodiment of the present disclosure is not limited thereto. The gaze range of the passenger and the facial angle of the passenger may be detected by the passenger monitoring system. The rotation angle of the passenger body and the distance between the passenger's hand and the door handle can be detected by posture detection, and the distance to the approaching vehicle coming from behind can be detected by object detection.If the value for the passenger's door opening intention S total is not greater than the door opening threshold T 1 and / or the distance to the vehicle approaching from behind is not less than the warning threshold T 2, then return to step S310. When the value of the door opening intention of the passenger S total is larger than the door opening threshold T 1 and when the distance to the approaching vehicle coming from behind is smaller than the warning threshold T 2, step S330 is executed to provide a warning of the opening of the door. In step S 330, the transparent display 170 may display the image of the vehicle coming from behind and illuminate a warning, the speaker 191 may output a warning sound, the LED 180 on the door interior may blink, and the external display 190 may display a warning that a door is being opened (e.g., in the form of text or images).In the present embodiment, the warning displayed on the transparent display 170 becomes clearer when the distance to the rear vehicle is shorter. For example, when the distance to the vehicle coming from behind is shorter, the image of the vehicle coming from behind on the transparent display 170 may be changed from small to large, and the transparency of the corresponding display region may be reduced to make the image of the vehicle coming from behind more recognizable.FIG. 4 is a schematic diagram showing an operation diagram of a display system for a side window on vehicles according to an embodiment of the present disclosure. In this regard, referring to FIGS. 1A, 1B, and 4, in this embodiment, besides the parking warning mode and the brightness setting mode which are executed permanently as shown in FIGS. 2 and 3, there are also application modes selected by the passengers. In this embodiment, for example, the transparent display 170 may display an augmented reality (AR) navigation mode icon 410, a game mode icon 420, an electronic whiteboard mode icon 430, and a video conferencing mode icon 440. In addition, the passenger may select the desired application mode by touch, i.e., the cockpit domain controller 110 may select the operation mode based on one of the plurality of operation icons touched by the passenger.FIG. 5A is a schematic diagram showing a process flow of an AR navigation algorithm of a display system for a side window on vehicles according to an embodiment of the present disclosure. See also FIGS. 1A, 1B and 5A. In this embodiment, the AR navigation algorithm includes two parts: an offline processing process 510 and an online processing process 520. In the offline processing process, the image captured by the external environment needs to be corrected with the gaze of the human. Moreover, object detection 512 for the external scene 511 and gaze tracking 514 for the passenger image 513 may be performed. Then, gaze correction 515 is performed based on the results of object detection 512 and gaze tracking 514. Among the above, object detection 512 is used to detect objects outside the vehicle window.In the online processing process 520, with the corrected model generated by the offline processing process 510, the TOI corresponding to the objects of the external environment may be mapped according to the eye and facial angles. Moreover, in addition to performing object detection 512 on the external scene 511 and performing gaze tracking 514 on the passenger image 513, object mapping 521 may be performed to map the gaze of the passenger and the detected objects outside the window of the car. Then, the position of the object (i.e., the TOI) assigned by the passenger is determined by the position information provided by the electronic compass 120, the angle of view of the passenger, and the position information provided by the positioning system 130, and is connected to the cloud system via the wireless communication module 140 to perform an object description search 522 via the cloud system, thereby searching for object information of the TOI. Then, when the microphone 163 receives the voice command transmitted from the voice 523 of the passenger, the voice recognition 524 may be performed on the voice 523 of the passenger to determine whether the voice 523 of the passenger transmits the voice command. When the voice command transmitted from the passenger's voice 523 refers to the TOI, a transparent display / speaker response 525 occurs, such as displaying text information via the transparent display 170 and / or a text-to-speech algorithm for converting the TOI's object information to voice to render the information via the speaker 191. Among the above, the position information provided by the electronic compass 120 and the position information provided by the positioning system 130 may be used to filter the searched content to improve search accuracy.FIG. 5B is a flowchart of the AR navigation mode of the display system for a side window on vehicles according to an embodiment of the present disclosure. See also FIGS. 1A, 1B, 4, 5A and 5B. In this embodiment, the side glass display system 100 may enter augmented reality navigation mode when the passenger touches the augmented reality navigation mode icon 410 on the transparent display 170. Under these circumstances, when the passenger looks out of the window, the side glass display system 100 may find the TOI information and place the TOI information on the transparent display 170, or the passenger may capture the (image) of the TOI and may combine the image of the TOI with the face captured by the in-car internal camera 161 to perform image reconstruction / composition.In addition, in step S510, it is determined whether the passenger selects the AR navigation function by touching. If the passenger does not touch to select the AR navigation function, the process is restarted; if the passenger touches to select the AR navigation function, step S520 is executed to execute the online processing process of the AR navigation algorithm. Next, in step S530, it is determined whether the passenger touches the return icon (e.g., return icon 546a in FIG. 5E). If the passenger does not touch the return icon, execution of the AR navigation algorithm is maintained; if the passenger touches the return icon, step S 540 is executed to return to the main menu (as shown in FIG. 4 ).FIG. 5C is a schematic diagram illustrating the AR navigation mode of the display system for a side window on vehicles according to an embodiment of the present disclosure. FIGS. 5D to 5G are schematic diagrams illustrating the operation in the AR navigation mode of the display system for a side window on vehicles according to an embodiment of the present disclosure. See also FIGS. 1A, 1B, 4, 5A to 5G. In this embodiment, the side-view display system 100, upon entering augmented reality navigation mode, uses an object detection algorithm to detect the object (marked by frames 541- 543 in FIG. 5D ) in the scene outside the view. Then, the passenger's TOI 532 (i.e., the object marked by the frame 543) can be found according to the gaze of the passenger 531. Upon receiving the voice command 529 of the passenger, in addition to displaying the text information 533 on the transparent display 170, the voice information 530 may also be reproduced via the speaker 191. As shown in FIG. 5E, the transparency of the area 544 displaying the text information 545 may be reduced and / or the transparency of the area 546 displaying the return icon 546 amay be reduced such that the text information 545 and the return icon 546 aare more recognizable to the passengers. In other words, the cockpit domain controller 110 may control the transparency distribution and brightness distribution of the transparent display 170 to reduce the transparency of at least one area on the transparent display 170 that displays text and / or symbols.As shown in FIGS. 5F and 5G, the object 543 amay be dragged when the passenger continues to press the object 543 ain the TOI 543 for longer. When the object 543 ais dragged, the total transparency of the transparent display 170 can be reduced, so that the dragged object 543 acan be clearly displayed and moved. Additionally, the passenger may select the portrait 547 or other objects to be displayed on the transparent display 170. When the object 543 ais pressed, an operation menu may appear so that the image of the object 543 acan be dragged, copied, stored, etc., which can be determined based on the system design, and the embodiment of the present disclosure is not limited thereto.According to the above, the cockpit domain controller 110 recognizes the TOI corresponding to the passenger in the external environment image based on the voice recognition result of the voice signal of the passenger, and determines the projection position of the TOI on the transparent display 170 based on the perspective of the passenger. Then, the cockpit domain controller 110 controls the transparency distribution and brightness distribution of the transparent display 170 to mark the projection position on the transparent display 170. In addition, the cockpit domain controller 110 controls the transparency distribution and brightness distribution of the transparent display 170 to display the TOI information next to the projection position on the transparent display 170.In an embodiment of the present disclosure, the cockpit domain controller 110 may draw one of these operation icons, a game icon, or a TOI area in the external image on the transparent display 170 depending on what the passenger presses.FIG. 5H is a flowchart of an online processing process of the AR navigation algorithm of the side window display system on vehicles according to an embodiment of the present disclosure. See also FIGS. 1A, 1B, 5A, 5B and 5H. In this embodiment, the online processing process of the AR navigation algorithm may include the following steps. In step S 521, the TOI may be automatically detected by the algorithm of object detection and gaze tracking. In step S522, it is determined whether the passenger long presses the object outside the disc. If the passenger does not push the object outside the pane long, step S521 is returned; if the passenger pushes the object outside the pane long, step S523 is executed. In step S523, after releasing the finger, the TOI is displayed on the transparent car window, and actions such as drag, copy, and storage can be performed.FIG. 6A is a schematic diagram showing the process flow of a game algorithm of a display system for a side window on vehicles according to an embodiment of the present disclosure. See also FIGS. 1A, 1B, 5A and 6A. In this embodiment, the game algorithm includes two parts: an offline processing process 610 and an online processing process 620. The description of the offline processing process 610 may be derived from the offline processing process 510 in the AR navigation algorithm in FIG. 5A, and details are not repeated here.In the online processing process 620, with the corrected model generated by the offline processing process 610, the game screen may be mapped to the external environment and the virtual character 621 according to the eye and facial angles. In addition to object detection 512 of external scene 511 and gaze tracking 514 of passenger image 513, game mapping 622 may be performed to map the passenger's gaze, detected objects outside the car window, and virtual character 621 to project the game screen. After the game screen is projected, posture recognition 624 is performed on the passenger image 623 to recognize the passenger's body movements. Subsequently, the game 620 may be controlled 625 by the passenger's body movements, gaze, and / or touch to control the movements of the characters.FIG. 6B is a flowchart of a game mode of a display system for a side window on vehicles according to an embodiment of the present disclosure. See also FIGS. 1A, 1B, 4, 6A and 6B. In this embodiment, the side-window display system 100 may transition to the gaming mode when the passenger touches the gaming mode icon 420 on the transparent display 170. In other words, when passengers wish some entertainment, they may use the external camera 151 to capture the outside-slice landscape, to recognize the gaze and gesture recognition of the passenger by the internal camera 161, and to combine the virtual character with the outside-slice landscape to achieve an integrated virtual and real entertainment.In addition, in step S610, it is determined whether the passenger selects the performance function by touching. If the passenger does not touch to select the performance function, the process is restarted; if the passenger touches to select the performance function, step S620 is executed to execute the online processing process of the performance algorithm. Next, in step S630, it is determined whether the passenger touches the return icon (return icon 546a in FIG. 5E). If the passenger does not touch the return icon, execution of the performance algorithm is maintained; if the passenger touches the return icon, step S640 is executed to return to the main menu (as shown in Fig. 4).FIG. 6C is a schematic diagram illustrating operation of the side window display system on vehicles in the game mode according to an embodiment of the present disclosure. See also FIGS. 1A, 1B, 4, 6A to 6C. In this embodiment, upon entering the game mode, the side-window display system 100 may capture the scene 631 outside the window via the external camera 151, perform edge recognition to define the edge 632 of the object in the background 631, and insert the virtual character 633 to be combined with the background 631, thereby achieving integrated virtual and real entertainment.FIG. 7 is a flowchart of an electronic whiteboard mode of the display system for a side window on vehicles according to an embodiment of the present disclosure. Also see FIGS. 1A, 1B, 4, and 7. in this embodiment, the side glass display system 100 may enter the electronic whiteboard mode when the passenger touches the electronic whiteboard mode icon 430 on the transparent display 170. Upon entering electronic whiteboard mode, the passenger may use the vehicle window as an electronic whiteboard / canvas via the touch panel 164 attached to the transparent display 170.In addition, in step S 710 it is determined whether the passenger selects the electronic whiteboard function by touching. If the passenger does not touch to select the electronic whiteboard function, the process starts again; if the passenger touches to select the electronic whiteboard function, step S720 is executed to call the electronic whiteboard side. Under these circumstances, the transparency of the transparent display is set to the minimum (i.e., it is made opaque). Next, in step S730, it is determined whether the passenger touches the return icon (return icon 546a in FIG. 5E). If the passenger does not touch the return icon, the electronic whiteboard execution is maintained; if the passenger touches the return icon, step S 740 is executed to return to the main menu (as shown in FIG. 4 ). Based on the above, when text or symbols are displayed on the transparent display 170, the cockpit domain controller 110 controls the transparency distribution and brightness distribution of the transparent display 170, so that the transparency of the entire transparent display 170 is reduced.FIG. 8 is a flowchart of a video conferencing mode of the side window display system on vehicles according to an embodiment of the present disclosure. In this regard, see also FIGS. 1A, 1B, 4, and 8. In this embodiment, the side glass display system 100 may enter the video conferencing mode when the passenger touches the video conferencing mode icon 440 on the transparent display 170. Upon entering the video conferencing mode, the passenger may perform a video conferencing via the in-vehicle camera and network functions.In addition, in step S810, it is determined whether the passenger selects the videoconferencing function by touching. If the passenger does not touch to select the video conferencing function, the process is restarted; if the passenger touches to select the video conferencing function, step S820 is executed to call the video conferencing page. Under these circumstances, the transparency of the transparent display 170 is set to the minimum (i.e., it is made opaque). Next, in step S 830, it is determined whether the passenger touches the return icon (return icon 546 ain FIG. 5E ). If the passenger does not touch the return icon, the execution of the video conference continues; if the passenger touches the return icon, step S840 is executed to return to the main menu (as shown in FIG. 4).FIG. 9 is a flow chart illustrating initiation of conference records in video conferencing mode of the side window display system on vehicles according to an embodiment of the present disclosure. In this regard, see also FIGS. 1A, 1B, 4, 8 and 9. In this embodiment, upon entering the video conference, recording of the verbal session minutes / videos can be started by voice commands.Moreover, in step S910, it is determined whether the passenger inputs a voice command for recording the meeting contents. If the passenger does not input the voice command for recording the meeting content, the process is restarted; if the passenger inputs the voice command for recording the meeting content, step S920 is executed for recording the meeting content. Subsequently, in step S930, it is determined whether the conference ends. If the conference is not finished, the execution of recording the meeting content continues; if the conference ends, step S940 is executed to display the meeting content on the transparent display 170.FIG. 10 is a flowchart showing the operation of triggering the display shutdown in the display system for a side window of vehicles according to an embodiment of the present disclosure. In this regard, see FIGS. 1A, 1B and 10. In this embodiment, the function of triggering the switch-off of the display can be executed permanently, i.e. at any time and everywhere. In other words, the passenger can turn off the transparent display 170 through the voice control function.Further, in step S1010, it is determined whether the passenger inputs a voice command for turning off the transparent display 170. If the passenger does not input the voice command for turning off the transparent display 170, the process starts again; if the passenger inputs the voice command for turning off the transparent display 170, step S 1020 is executed to turn off the transparent display 170. Based on the foregoing, the cockpit domain controller 110 may determine whether the voice signal is a command voice according to the passenger, and when the voice signal is a command voice, the cockpit domain controller 110 selects an operation mode based on the command voice.FIG. 11 is a flowchart illustrating the operation of triggering the passenger sleep mode of the side window display system on vehicles according to an embodiment of the present disclosure. In this regard, see FIGS. 1A, 1B, and 11. In this embodiment, the function for triggering the passenger sleep mode may be executed permanently, i.e., anytime and anywhere. In other words, when the passenger slaps, the vehicle side window display system 100 may automatically turn off the content of the transparent display 170 (e.g., display the content in pure white or pure black) and reduce the transparency.In addition, it is determined in step S1110 that the passenger has fallen sleep and the sleep has already continued for a certain time (i.e., >N seconds), where N is a positive integer. If the passenger slaps for less than N seconds, the process is restarted; if the passenger slaps for more than N seconds, step S1120 is executed to turn off the transparent display 170 and reduce the transparency.When the cockpit domain controller 110 determines that the passenger is sleeping based on the eye angle and posture of the passenger based on the above, and when the sleep continues for a predetermined period of time, the cockpit domain controller 110 controls the transparency distribution and brightness distribution of the transparent display 170 to reduce the transparency of the transparent display 170.In summary, in the side glass display system according to the embodiment of the present disclosure, the cockpit domain controller may acquire the current environmental information inside and outside the vehicle via the sensors disposed inside and outside the vehicle to control the transparency and brightness setting of the transparent display. In this way, it is possible to allow passengers to have a virtual and real interactive experience with the image outside the pane, thereby improving the immersive experience and increasing the applicability of vehicles.[Description of Reference Numerals]10 Vehicle 85 sky tower 100 side window display system 110 cockpit domain controller 120 electronic compass 130 positioning system 140 wireless communication module 150 external sensor 151 external camera 152 external light sensor 160 internal sensor 161 internal camera 162 internal light sensor 163 microphone 164 touch panel 170 transparent display 180 light emitting diode (LED=light emitting diode) for a door interior 190 external display 191 speaker 410 augmented reality (AR) navigation mode icon 420 game mode icon 430 electronic whiteboard mode icon 440 video conferencing mode icon 510, 610 offline processing process 520, 620 Online processing process 521 Object association 522 Object description search 523 Voice of a passenger 524 Voice recognition 525 Response through the transparent screen / speaker 529 Voice command of a passenger 530 Voice information 531 Gaze of a passenger 532 TOI (Token Of Interest) of a passenger 541- 543 Frame 543 a Objekt 544, 546 Areas 545 Text information 546 a Return icon 620 Control the game 621 Virtual game character 622 Game association 623 Image of a passenger 624 Posture recognition 631 Background 632 Edge 633 Virtual game character S 210, S 220, S 230, S 240, S 304, S 310, S 320, S 330, S 510, S 520, S 521, S 522, S 523, S 530, S540, S610, S620, Step S630, S640, S710, S720, S730, S740, S810, S830, S840, S910, S920, S930, S940, S1010, S1020, S1110, S1120 Step

Claims

A display system (100) for a side window of a vehicle, comprising: a transparent display (170) disposed on a side window of a vehicle (10); at least one external sensor (150) configured to sense the external environment of the vehicle (10) to provide information about the external environment; at least one internal sensor (160) configured to sense an internal environment of the vehicle (10) to provide information about the internal environment; a controller (110) coupled to the transparent display (170), the at least one external sensor (150), and the at least one internal sensor (160) and configured to select an operation mode based on the information about the internal environment, and to control a transparency distribution and a brightness distribution of the transparent display (170) based on the operation mode and the information about the external environment.The side glass display system (100) of claim 1, wherein the at least one internal sensor (160) includes an internal light sensor (162) configured to detect an internal light brightness inside the vehicle (10), and wherein the at least one external sensor (150) includes an external light sensor (152) configured to detect an external light brightness outside the vehicle (10), wherein the controller (110) is configured to control the transparency distribution and the brightness distribution of the transparent display (170) based on a brightness error of the internal light brightness and the external light brightness.The side glass display system (100) according to claim 1 or 2, wherein the at least one internal sensor (160) includes an internal camera (161) configured to detect a viewing angle of eyes of a passenger and a posture of the passenger.The side glass display system (100) according to claim 3, wherein when the controller (110) determines that the vehicle (10) is in a parking state, the controller (110) is configured to determine whether the passenger is about to open a door and whether a car is approaching the vehicle (10) from behind, wherein when the controller (110) determines that the passenger is about to open the door and determines that the car is approaching the vehicle (10) from behind, the controller (110) is configured to control the transparency distribution and the brightness distribution of the transparent display (170) to display a warning message about an approaching car on the transparent display (170).The side glass display system (100) according to claim 3 or 4, wherein the at least one external sensor (150) includes an external camera (151) for capturing an external environment image outside the vehicle (10), wherein the at least one internal sensor (160) includes a microphone (163) for capturing a voice signal of the passenger, wherein the controller (110) is configured to capture a target of interest, TOI, corresponding to the passenger in the external environment image based on a voice recognition result of the voice signal, and determine a projection position of the TOI on the transparent display (170) based on the viewing angle, and wherein the controller (110) is configured to control the transparency distribution and the brightness distribution of the transparent display (170), to mark the projection position on the transparent display (170).The display system (100) for a side glass according to claim 5, wherein the controller (110) is further configured to control the transparency distribution and the brightness distribution of the transparent display (170) to display TOI information adjacent to the projection position on the transparent display (170).The side glass display system (100) according to any one of claims 3 to 6, wherein when the control unit (110) determines that the passenger has fallen, based on the angle of view of the passenger's eyes and the posture of the passenger, and when the passenger has fallen for a period of preset time, the control unit (110) is configured to control the transparency distribution and the brightness distribution of the transparent display (170) to reduce transparency of the transparent display (170).The side glass display system (100) of any preceding claim, wherein the at least one internal sensor (160) comprises a touch panel (164) to detect that a passenger touches or presses the transparent display (170), wherein the controller (110) is configured to select the operation mode based on one of the operation icons touched by the passenger, and wherein the controller (110) is configured to draw one of the operation icons, a game icon, or a TOI area into an external image based on a pressing of the passenger on the transparent display (170).The display system (100) for a side glass according to any one of the preceding claims, wherein the controller (110) is configured to control the transparency distribution and the brightness distribution of the transparent display (170) so as to reduce transparency of at least one area (544, 546) displaying a text or a symbol on the transparent display (170).The display system (100) for a side glass according to any one of the preceding claims, wherein when text or symbols are displayed on the transparent display (170), the controller (110) is configured to control the transparency distribution and the brightness distribution of the transparent display (170) so as to reduce transparency of the entire transparent display (170).

Images