Windshield with partitioned display

Abstract

The system may include one or more displays. A system may include a pane of a vehicle including one or more regions wherein: a barrier is formed by the pane and separates an interior of the vehicle from an exterior of the vehicle, the barrier is one or more of transparent or translucent for all transmission of visible light from the exterior of the vehicle through the one or more regions to the interior of the vehicle, an outer periphery of the pane forming an edge and defining a surface area of the pane, and the one or more regions extend over more than 20% of the surface area. A system may cause a visual output to adjust the transmission of visible light through the one or more regions, wherein the visual output is based on the one or more displays.

Description

Technical Field

[0001] The present disclosure relates to a windshield having a zoned display and a vehicle including a windshield having a zoned display. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] During travel, vehicle occupants typically desire to observe or adjust their appearance. With the evolution of the appearance and feel of modern vehicles, by removing structural components previously used to address occupant grooming purposes, the surface area of transparent or translucent structural components can be increased to enhance the occupant experience. Summary of the Invention

[0004] This section provides an overview of the present disclosure and not an exhaustive disclosure of its full scope or all of its features.

[0005] For example, a vehicle may remove support structures (e.g., body panels, crossbars, pillars) to provide a more transparent passenger compartment. For example, the windshield and roof may be formed by a single pane or formed without opaque support structures, which can eliminate the support structures necessary for sun visors, vanity mirrors, and other vehicle components. Cantilever supports or other mechanisms may provide access to sun visors, vanity mirrors, and other vehicle components and further degrade the occupant experience by blocking the view through the pane (e.g., windshield, window). A display may be used to provide an indication to the occupant of their current appearance or provide other information or entertainment content without blocking the view with opaque components.

[0006] In one or more forms, the techniques described herein relate to a vehicle including: one or more displays; a pane of the vehicle including one or more zones, wherein: a barrier is formed by the pane, and the barrier separates the interior of the vehicle from the exterior of the vehicle, the barrier being transparent or translucent, or a combination thereof, for all transmissions of visible light from the exterior of the vehicle through the one or more zones to the interior of the vehicle, an outer perimeter of the pane forms an edge of the pane and defines a surface area of the pane, and the one or more zones extend over more than 15% of the surface area; one or more processors; and a non-transitory memory including instructions that, when executed by the one or more processors, are operable to: cause a visual output to adjust the transmission of visible light through the one or more zones, wherein the visual output is based on the one or more displays.

[0007] In one or more forms, the techniques described herein relate to a vehicle, where the one or more displays include a first display and a second display, the first display being configured to provide a first portion of the visual output based on a projector, and the second display being configured to provide a second portion of the visual output based on an organic light-emitting diode integrated with the pane. The one or more regions include a first region and a second region, and the first display is configured to provide the first portion of the visual output within the first region, and the second display is configured to provide the second portion of the visual output within the second region. The first region forms a boundary with the second region, and the brightness of the first portion of the visual output at the boundary is less than the maximum brightness of the first portion of the visual output. The first region forms a boundary with the second region, and the brightness of the first portion of the visual output at the boundary is less than the maximum brightness of the second portion of the visual output. The first display includes a wedge film, and the wedge film is configured to reduce the occurrence of reflections associated with the first display or the second display. The vehicle may include a support structure of the vehicle. The support structure forms an inner perimeter, and the pane is sized to extend to the inner perimeter and the inner perimeter engages the outer perimeter of the pane. The visual output is based on a digital representation and the digital representation depicts the surrounding environment outside the vehicle. The surrounding environment outside the vehicle includes a rear view, and the rear view is based on a perspective from a position on the vehicle. The vehicle includes a projector of the one or more displays disposed on the roof of the vehicle. The position is on one side of the vehicle. The visual output increases the opacity of the barrier. During a first operating state, the one or more regions extend over less than 50% of the surface area, and during a second operating state, the one or more regions extend over more than 50% of the surface area. The one or more displays include a first display based on frit, a second display based on a projector, and a third display based on an organic light-emitting diode. One or more sensors are configured to output one or more digital representations, and the one or more displays include a first display and a second display, the first display being configured to provide a first portion of the visual output, and the second display being configured to provide a second portion of the visual output. The one or more regions include a first region and a second region, and the first display is configured to provide the first portion of the visual output within the first region, and the second display is configured to provide the second portion of the visual output within the second region. The visual output includes a first user interface element within the first portion of the visual output and a second user interface element within the second portion of the visual output. The non-transitory memory further includes a neural network and instructions that, when executed by the one or more processors, are operable to identify one or more poses. The execution causes the identification of the one or more poses to be based on the one or more digital representations and the neural network. The execution causes the determination of the operation.The determination of the operation is based on the one or more gestures and the neural network. The execution results in the determination of a selection. The determination of the selection indicates the first user interface element or the second user interface element. The determination of the selection is based on the one or more gestures, the one or more neural networks, and the execution of the operation. The execution of the operation is based on the one or more processors and the selection. The one or more sensors include a first sensor positioned on the roof of the vehicle and a second sensor positioned on the dashboard of the vehicle, the one or more digital representations include a first digital representation based on the first sensor and a second digital representation based on the second sensor, and a first input of the neural network is based on the first digital representation and a second input of the neural network is based on the second digital representation.

[0008] In one or more forms, the techniques described herein relate to a method for operating a vehicle, the method including causing a visual output to adjust the transmission of visible light through a first region and a second region. The visual output includes a first user interface element and a second user interface element. The first user interface element is depicted within the first region and the second user interface element is depicted within the second region. The method includes identifying one or more gestures. The identification of the one or more gestures is based on one or more digital representations. The one or more gestures indicate an operation and a selection, and the selection indicates the first user interface element or the second user interface element. The method includes executing the operation, wherein the execution of the operation is based on the selection. The operation includes an adjustment to the experience provided to an occupant of the vehicle.

[0009] In one or more forms, the techniques described herein relate to a method for training a neural network, the method including managing training data, wherein the training data includes depictions of gestures and annotations of those gestures, and the gestures are based on depictions of the position of a first body part and depictions of the movement of a second body part. The method includes training the neural network based on a first subset of the training data. The neural network is configured to identify an operation and a selection of a first user interface element or a second user interface element. The method includes validating the neural network based on a second subset of the training data.

[0010] In one or more forms, the techniques described herein relate to a method, wherein the first body part includes one or more of an eye, a head, or a neck, and the second body part includes a hand.

[0011] From the description provided herein, further applicability will be apparent. It should be understood that the description and specific examples are intended for illustrative purposes only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] To better understand the present disclosure, various forms of the present disclosure given by way of example will now be described with reference to the accompanying drawings, in which:

[0013] Figure 1 A system according to one or more implementations of the present disclosure is shown;

[0014] Figure 2A One or more portions of a display according to one or more implementations of the present disclosure are shown;

[0015] Figure 2B One or more portions of a display according to one or more implementations of the present disclosure are shown;

[0016] Figure 3 A schematic diagram of a system according to one or more implementations of the present disclosure is shown;

[0017] Figure 4 A multi-pane display according to one or more implementations of the present disclosure is shown;

[0018] Figure 5A A gaze according to one or more implementations of the present disclosure is shown;

[0019] Figure 5B Another gaze according to one or more implementations of the present disclosure is shown;

[0020] Figure 5C Yet another gaze according to one or more implementations of the present disclosure is shown;

[0021] Figure 6A A digital representation according to one or more implementations of the present disclosure is shown;

[0022] Figure 6B A visual representation according to one or more implementations of the present disclosure is shown;

[0023] Figure 7 An operating state and weather conditions according to one or more implementations of the present disclosure are shown;

[0024] Figure 8 An operating state and weather conditions according to one or more implementations of the present disclosure are shown;

[0025] Figure 9 A method for providing a visual representation according to one or more implementations of the present disclosure is shown; and

[0026] Figure 10 A method for providing a visual representation according to one or more implementations of the present disclosure is shown; and

[0027] Figure 11 shows a method for providing a visual representation according to one or more implementations of the present disclosure; and

[0028] Figure 12 shows a method for conducting a teleconference according to one or more implementations of the present disclosure.

[0029] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way. Detailed Description

[0030] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses. It should be understood that throughout the drawings, corresponding reference numerals indicate the same or corresponding parts and features.

[0031] In one or more forms, the present disclosure includes a method for depicting a visual representation on one or more panes of a vehicle. The one or more panes include a first location. The method includes determining the visual representation based on an occupant. The method further includes depicting the visual representation at the first location. The determination of the visual representation may include capturing radiation reflected from the occupant. The determination of the visual representation may include applying a transformation to a digital representation based on the radiation. The transformation may adjust perspective distortion of the digital representation. The radiation may be within the visible spectrum, or the radiation is within the infrared spectrum.

[0032] Reference Figure 1 , shows a system 100 according to one or more implementations of the present disclosure. System 100 includes one or more panes 102. The pane 102 may be a windshield, a side window, a rear window, or another transparent or translucent component that allows an occupant to view the external environment of the vehicle. The pane 102 may form a barrier between the interior and the exterior of the vehicle, and the pane 102 may allow transmission of light visible to a typical human eye. The visible light may be enhanced by a display (e.g., display 104). The pane 102 may include an edge 134 that forms the outer perimeter of the pane. The outer perimeter of the pane 102 may engage an inner perimeter of a support structure of the vehicle (e.g., a body panel, a crossbar, a pillar). The pane 102 includes one or more layers of glass, plastic, or other components that constitute the display 104.

[0033] For example, the pane 102 may be configured to allow augmented reality for an occupant across the pane 102. The pane 102 may include technology for providing augmented reality in the form of a heads-up display. The heads-up display may provide information, indications, representations, graphics, and other depictions without the gaze associated with the occupant leaving the pane 102. Some example technologies for providing the display 104 are described herein, and those technologies described herein are not an exhaustive list of technologies contemplated for providing augmented reality to an occupant via a heads-up display. The display 104 may cause a visual output. The visual output may include one or more user interface elements 130, 132 or the visual representations 120 discussed herein. The user interface elements 130, 132 may be used to interact with the vehicle or other systems. For example, the user interface elements 130, 132 may be depicted as knobs, switches, buttons, or another control for performing an operation (e.g., playing a movie, adjusting the volume, changing the air conditioning, locking the doors). The visual output may include content (e.g., video, images, graphics) or any other light emission within the electromagnetic spectrum or perceptible to the human eye.

[0034] The display 104 includes at least one region (e.g., regions 106, 108, 110, 112, 114, 116, 118) for depicting information (e.g., one or more portions 122, 124 of the visual representation 120) on the pane 102 such that light passing through the pane 102 is transmitted to the eyes of the occupant. The transmission of light may be enhanced to provide augmented reality to the occupant. The visual representation may be based on the vehicle's occupant, a participant in a conference call, or a combination thereof. The regions 106, 108, 110, 112, 114, 116, 118 may be defined by positions associated with particular display technologies. For example, regions near the dashboard (e.g., regions 112, 114, 116) may be provided by a projector-based heads-up display or otherwise, and regions near the top of the pane 102 or on the roof portion of the pane 102 (e.g., regions 108, 110, 118) may be provided by technologies based on organic light-emitting diode (OLED) arrays, liquid crystal displays, transparent displays, micro LEDs, neoQLEDs, or otherwise. The outputs from the heads-up display technologies may be integrated such that the display 104 fills the entire pane or a portion of the pane. The regions 106, 108, 110, 112, 114, 116, 118 are shown as various shapes and sizes and integrated together as a patchwork such that the display provides a desired coverage area. These regions may have adjacent boundaries such that the depiction of the visual representation (e.g., visual representation 120) is seamless or the occupant cannot perceive that the depiction is provided by different display technologies. Region 106 is used to provide blind spot monitoring and may be similarly located on the driver side or passenger side of the vehicle.

[0035] System 100 includes sensors 126 (e.g., visible light cameras, infrared detectors) for generating visual representation 120. For example, sensors 126 may capture visible light (e.g., electromagnetic radiation 128) generated by display 104 and reflected from an occupant. Sensors 126 may convert electromagnetic radiation 128 from energy into digital values that may indicate a representation of the occupant (e.g., visual representation 120). Visual representation 120 is shown depicted at a first position 140. First position 140 may have a greater vertical height relative to occupant 320 (occupant 320 is shown in Figure 3 than a second position 142. Visual representation 120 may move from one position to another based on input, eye gaze, pose, other information, or a combination thereof.

[0036] Reference Figure 2A , one or more portions of display 104 are shown in accordance with one or more implementations of the present disclosure. As described herein, display 104 may include one or more technologies for providing augmented reality to an occupant. For example, display 104 may be generated based on projector 202. For example, display 104 may be based on a technique that reflects light 204 emitted from projector 202. Projector 202 may be configured to emit light 204 toward one or more mirrors 206, 208 to depict a representation (e.g., visual representation 120) or a portion of a representation in one or more areas (e.g., areas 112, 114, 116). Light 204 emitted by projector 202 may be reflected from pane 102 to produce display 104 visible to a vehicle occupant in a passenger seat.

[0037] Reference Figure 2B , one or more portions of display 104 are shown in accordance with one or more implementations of the present disclosure. As described herein, display 104 may include one or more technologies for providing augmented reality to an occupant. For example, display 104 may be generated based on one or more light emitting diodes 220, liquid crystals, plasma pixels, or other technologies. For example, display 104 may be based on a technique embedded in pane 102 for viewing by a vehicle occupant in a passenger seat. As shown, light emitting diodes 220 are distributed across pane 102 to provide display 104 in an area (e.g., area 118).

[0038] Adjacent technologies may suffer from overlap or blurring caused by bleed or reflection of adjacent technologies. Wedge films may be used to reduce overlap or blurring between the edges of adjacent regions. Additionally, darkening (e.g., reducing brightness) of the boundary regions at the adjacency of regions may be used to reduce overlap, blurring, bleed, unwanted reflections, or other defects caused by adjacent technologies.

[0039] Reference Figure 3, shows a schematic diagram of a system 100 according to one or more implementations of the present disclosure. The system 100 includes a sensor 126. The sensor 126 can be configured to capture electromagnetic radiation 128. The sensor 126 provides an indication of the electromagnetic radiation 128 reflected from the occupant 320 to the controller 300. The electromagnetic radiation can originate from ambient light or the display 104. The sensor 126 can be located on the vehicle's dashboard or console. The sensor 136 can be located on the vehicle's roof, crossbeam, or pillar.

[0040] The sensor 126 can convert the electromagnetic radiation 128 into a digital form and communicate with the controller 300 via a communication bus. The communication bus can be a Controller Area Network (CAN). The controller 300 can include one or more processors 306, a non-transitory memory 302, and instructions 304 disposed thereon. The instructions 304 can be configured to perform one or more steps (e.g., determine, depict, transform) described herein when executed by the one or more processors 306. For example, the instructions 304 can cause the one or more processors 306 to output a visual representation 120 from the projector 202 or the light-emitting diode 220 for depiction on the display 104.

[0041] Reference Figure 4 , shows a display 104 across multiple panes 102, 402 according to one or more implementations of the present disclosure. Similar to the description regarding Figure 1 , the display 104 can occupy multiple panes of the vehicle 400. For example, the display 104 can include multiple regions (e.g., regions 118, 404) distributed across the first pane 102 and the second pane 402. The second pane 402 can operate similarly to the first pane 102. The visual representation 120 can be depicted across one or more regions 118, 404. The region 404 can operate similarly to the region 118. The visual representation 120 can be moved to different positions within the display 104 (e.g., between the regions 118, 404) based on the gaze of the occupant 320 as discussed herein.

[0042] Reference Figures 5A to 5C, which shows fixations 502, 504, 506 according to one or more implementations of the present disclosure. The fixations 502, 504, 506 can be determined relative to the eyelids, head position, neck position, or another body position. For example, the position of the eye pupil can indicate the field of view of the occupant 320. Based on the field of view or fixation (e.g., fixations 502, 504, 506), the controller 300 can be configured to adjust the position of the visual representation 120. For example, the controller 300 can adjust the visual representation 120 from a first region to a second region. The controller 300 can adjust the visual representation 120 from a first position within a region (e.g., region 116) to a second position. This position can be defined based on a coordinate system, where the position of the display 104 is mapped to a position on the pane 102. For example, the position or orientation of the visual representation can be based on the fixation of the occupant 320 (e.g., fixations 502, 504, 506).

[0043] Reference Figures 6A to 6B , which shows the digital representation 604 and the visual representation 120 according to one or more implementations of the present disclosure. The digital representation 604 can be an image (e.g., JPEG) based on data from sensors (e.g., sensors 126, 406) determined according to radiation (e.g., radiation 128). The position of the sensors can skew the digital representation 604. For example, a sensor (e.g., sensor 126) mounted near the console causes the digital representation to be from a perspective that distorts the occupant. A sensor (e.g., sensor 406) mounted on the door structure can also cause the digital representation to distort the occupant. Thus, a transformation can be applied to change the perspective of the digital representation 604 and the associated pixels 602 to a forward-facing visual representation 120 and pixels 606 for display on one or more panes 102, 402. The transformation can be a per-pixel mapping configured to distort the perspective of the digital representation based on the orientation and position of the sensors (e.g., sensors 126, 406). That is, the transformation can be specific and based on vehicle model, sensor position, occupant position, fixation, pane position, or other factors.

[0044] Reference Figure 7, showing the operating state and weather conditions according to one or more implementations of the present disclosure. The visual representation 120 can be displayed based on the operating state of the vehicle 700. For example, during periods when the vehicle speed is greater than a threshold, the visual representation 120 can be adjusted to avoid interfering with driving or other interactions. The operating state of the vehicle 700 can be stored as data on one or more non-transitory memories. For example, when the vehicle speed is greater than the threshold, the visual representation 120 can be removed from the display 104, while when the vehicle speed is less than the threshold or stopped, the visual representation 120 can be displayed on the display 104. Additionally, the size of the visual representation 120 can be based on the vehicle speed, or the position of the visual representation 120 can be adjusted based on the vehicle's speed, direction, or other operating states. The operating state can be based on a variety of vehicle parameters (e.g., speed, shifter position, road conditions, weather conditions). The weather conditions 702 around the vehicle can further affect the visual representation 120. For example, the brightness of the visual representation 120, the display 104, or other lighting devices (e.g., dome lights) can be based on the weather conditions 702. During inclement weather or night driving, the brightness can be reduced or adjusted. Additionally, sensors (e.g., sensors 126, 406) can be configured to operate in an infrared mode to require less ambient light to generate the digital representation 604 or the visual representation 120. The transformation can also utilize filters or other tools to enhance the digital representation 604 to alter the appearance of the occupant 320. For example, the filter can remove wrinkles, blemishes, or other cosmetic issues associated with the occupant 320. As Figure 7 shown, the vehicle 700 can have a surrounding environment 708. The surrounding environment 708 can be depicted with visual output. For example, a view 706 from the rear of the vehicle 700 can be shown on the display 104. A view 704 from the side of the vehicle 700 can be shown on the display 104.

[0045] Reference Figure 8 , showing a neural network 800 according to one or more implementations of the present disclosure. The neural network 800 can be trained with managed data 802, 804, and annotations (e.g., ground truth). For example, the annotations can indicate the pose or eye gaze for a particular image or image stream. The managed training data can include a first image corpus that includes depictions representing the positions of body parts indicating the gaze of an occupant. For example, the first image corpus can include images of eyes, heads, necks, or other body parts related to the position of the eyes. The first image corpus can include annotation data indicating relevant gaze information (e.g., direction).

[0046] The managed data 802, 804 may include a second image corpus that includes depictions of movement of a second body part. For example, the movement may be based on an eye, a hand, or another body part indicating a desired action. The neural network 800 may include an input layer 806 for receiving images. The input layer may receive images or image streams from the managed training data 802, 804 during training or from sensors 126, 136, 406 during use in the vehicle to identify a pose, an action, or a selection. The input layer 806 may be cascaded in layer 808 and fed, either alone or with other data, to a feature recognition layer 810. The feature recognition layer 810 may be used to identify features within an image or digital representation to identify one or more poses. The pose may indicate an action 812 (e.g., turning a knob, pressing an augmented reality button). The action may increase the volume, take a photo, initiate a call, or otherwise provide an interface for an occupant of the vehicle to interact with the vehicle based on the display 104. The pose may also indicate a selection of one or more user interface elements across the display 104 or pane 102. For example, the pane 102 may be augmented to provide the display 104 across an entire windshield, window, or elsewhere, and a combination of the eyes and the pose may be used to control the vehicle using the user interface elements 130, 132. Using the user interface elements 130, 132 across the entire pane 102, a gaze may be used to determine an intended selection 814 between the user interface elements 130, 132, and a hand movement may be used to indicate the desired action 812. The action 812 and the selection 814 may be executed 816 by one or more processors to obtain a desired effect. During training, the error between the annotation 818 of the ground truth and the identified action 812 and selection 814 may be used to further improve the recognition of the neural network 800 until an acceptable error is obtained.

[0047] Reference Figure 9 , a method 900 is shown in accordance with one or more implementations of the present disclosure. The method 900 may provide an augmented reality experience for an occupant of the vehicle 700. One or more steps of the steps of the method 900 may be omitted, rearranged, or repeated. In step 902, a visual representation 120 may be determined based on the occupant 320. The occupant 320 may be a passenger or a driver of the vehicle 700. The visual representation 120 may be determined based on the techniques described herein or other techniques. The visual representation 120 may be based on radiation reflected from the occupant 320. For example, the visual representation 120 may be based on electromagnetic radiation (e.g., radiation 128), visible light, infrared light, or other light reflected from the occupant 320. The electromagnetic radiation may originate from the sun (e.g., ambient light) or other light sources (e.g., dome lighting) or light emitted from the augmented reality display 104.

[0048] Step 902 may include additional steps for adjusting the appearance of the visual representation 120. For example, sensors (e.g., sensors 126, 406) may capture electromagnetic radiation (e.g., radiation 128) indicative of the occupant 320. The sensors may convert the electromagnetic radiation into a digital representation 604. The digital representation 604 may be skewed from the sensor orientation relative to the occupant 320 and the display orientation. Accordingly, the digital representation 604 and associated pixels 602 may be transformed from a first perspective to a second perspective to form pixels 606. This transformation may warp the pixels 602 to the pixels 606 to form the visual representation 120 such that the pixels 606 of the visual representation 120 are displayed as appearing from a different perspective than the perspective from which the pixels 602 were captured. In this manner, the sensors (e.g., sensors 126, 406) may be positioned to capture electromagnetic radiation from a different perspective than the electromagnetic regulation emitted from the display 104.

[0049] In step 904, the visual representation 120 may be depicted. For example, the display 104 may be configured to present the visual representation 120 in one or more regions of one or more panes 102. For example, more than one display technology may be used to depict the visual representation 120. The depiction may be based on one or more inputs, and the depiction may include various parameters or settings. For example, the parameters may define how the visual representation 120 is depicted (e.g., position, size, brightness, filter) or when the visual representation 120 is depicted (e.g., based on the operating state of the vehicle 700). The inputs may be used to determine the parameters.

[0050] The input may be a switch actuation (e.g., button press), received from another device, determined based on the state of the vehicle or the vehicle's surroundings, or otherwise. The input may be information about an operation available to the vehicle 700 for affecting the depiction of the visual representation 120. The input may be the pose of the occupant 320. The pose may be determined by the controller 300 or sensors (e.g., sensors 126, 406). Artificial intelligence may be used to determine the pose. For example, a convolutional neural network may be used to determine the presence of a particular pose. The convolutional neural network may be trained on an image or video of the pose. The pose may be a physical movement of the occupant 320. For example, the pose may be a facial expression. The facial expression may include eye movement or a combination of hand or eye movement. For example, the occupant 320 may touch their face or look up as if looking in a mirror, which may trigger the depiction of the visual representation 120 and allow the occupant 320 to examine their face, eyes, hair, other features, or features associated with them personally (e.g., clothing). Gaze (e.g., gazes 502, 504, 506) may be monitored to determine the gaze direction and determine whether the occupant 320 is looking forward or up to be used as an input.

[0051] In step 906, the depiction can be adjusted from a first position to a second position. For example, the display 104 can define a coordinate system (e.g., Cartesian coordinates) relative to one or more panes 102. The visual representation 120 can be adjusted from the first position to the second position. The positions can be associated with regions 106, 108, 110, 112, 114, 116, 118 or be defined as subsets of these regions. These positions can be defined by the origin (e.g., lower left corner, upper right corner) of the visual representation 120. Exemplary first position 140 and exemplary second position 142 are shown in Figure 1 . The visual representation 120 can be moved from the first position 140 to the second position 142 based on the gaze of the occupant 320 (e.g., gazes 502, 504, 506) such that the visual representation 120 is depicted within the field of view of the occupant 320.

[0052] Radiation (e.g., radiation 128) received by sensors (e.g., sensors 126, 406) can be generated based on the depiction of the visual representation. For example, the depiction can emit visible light that can reflect off the occupant 320 and be further received or captured by sensors (e.g., sensors 126, 406).

[0053] In Figure 10 , a method 1000 according to one or more implementations of the present disclosure is shown. One or more of the steps of method 1000 can be omitted, rearranged, or repeated. In step 1002, a digital representation (e.g., digital representation 604) can be determined. For example, sensors (e.g., sensors 126, 406) can capture radiation (e.g., radiation 128) to define the digital representation 604. In step 1004, a conference call can be established. The conference call can use the Voice over IP protocol or another protocol to establish digital communication. The digital representation 604 can be sent to the participants of the conference call (e.g., the other end of the line). The participants can send their own digital representations, and the digital representations of the participants can be depicted as visual representations 120 on one or more panes 102 having the display 104. For example, the vehicle 700 can use the display 104 to provide augmented reality to the occupant 320. The conference call can use the filters or transformations described herein to depict the participants or the occupant 320. For example, the filter can remove wrinkles, blemishes, or other cosmetic issues associated with the representation of the participants or the representation of the occupant. The visual representation 120 can be the visual representation of the occupant 320 or the participants, and all the techniques described herein apply to the visual representation of the occupant 320 or the visual representation of the participants. In step 1006, the display 104 can be further configured to simultaneously (e.g., side by side) depict the visual representations of the occupant or the participants.

[0054] In Figure 11Shown therein is method 1100 according to one or more implementations of the present disclosure. The steps of method 1100 may be omitted, rearranged, repeated, or combined with any other steps described herein. Method 1100 may be used to operate a vehicle. For example, the vehicle may provide a visual output that changes the transmission of visible light through pane 102. The visual output may include user interface elements 130, 132, video, other elements, or graphics, or a combination thereof. User interface elements 130, 132 may be depicted in different regions 110, 112.

[0055] In step 1102, a visual output may be caused. The visual output may be based on display 104, pane 102, or a combination thereof. In step 1104, a pose may be recognized. For example, neural network 800 may be used to recognize one or more poses, and the pose may be used to determine operation 812 or selection 814. Selection 814 may indicate one or more user interface elements 130, 132. For example, user interface element 130 may be a depiction of a knob, and the pose may be used to select the knob and perform operation 812 associated with the knob (e.g., adjust the volume). In step 1106, operation 812 may be performed. For example, the volume may be adjusted based on the pose. Any operation that affects the vehicle experience or operation is contemplated.

[0056] In Figure 12 Shown therein is method 1200 according to one or more implementations of the present disclosure. Method 1200 may be used to train a neural network (e.g., neural network 800). The steps of method 1200 may be omitted, rearranged, repeated, or used in any other method described herein. In step 1202, training data 802, 804, and annotation 818 of the ground truth are managed. For example, training data 802, 804 may include depictions of poses and annotations of the poses. The poses may include depictions of eyes, hands, heads, necks, and other body parts that may be used to recognize selection 814 and operation 812. Images or image streams may be used to determine selection 814 and operation 812. The managed training data 802, 804 may be divided into two data subsets. The first subset may be used for training in step 1204. For example, the error between annotation 818 of the ground truth during training and the output of neural network 800 may be used to update the weights of neural network 800 (e.g., backpropagation) to improve accuracy and reduce loss. In step 1206, the weights may be updated (trained) until neural network 800 is verified to be above a predetermined loss using the second subset.

[0057] Unless otherwise expressly indicated herein, all numerical values representing mechanical / thermal properties, percentages of composition, dimensions and / or tolerances, or other characteristics, when describing the scope of the present disclosure, should be understood to be modified by the word "about" or "approximately". Such modification is required for various reasons, including industrial practices, materials, manufacturing and assembly tolerances, and testing capabilities.

[0058] As used herein, the phrase "at least one of A, B, and C" should be construed to represent the logic (A or B or C) using a non-exclusive logical "or", and should not be construed to mean "at least one of A, at least one of B, and at least one of C".

[0059] In the present application, the terms "controller" and / or "module" may refer to, be part of, or include the following, such as in a system-on-chip: application specific integrated circuit (ASIC); digital, analog, or mixed analog / digital discrete circuits; digital, analog, or mixed analog / digital integrated circuits; combinational logic circuits; field programmable gate array (FPGA); processor circuits (shared, dedicated, or grouped) that execute code; memory circuits (shared, dedicated, or grouped) that store the code executed by the processor circuits; other suitable hardware components that provide the function (e.g., an operational amplifier circuit integrator as part of a heat flux data module); or a combination of some or all of the above.

[0060] The term memory is a subset of the term computer-readable medium. As used herein, the term "computer-readable medium" does not cover transient electrical or electromagnetic signals propagated through a medium (such as on a carrier wave); the term computer-readable medium can thus be considered tangible and non-transitory. Non-limiting examples of non-transitory tangible computer-readable media are non-volatile memory circuits (such as flash memory circuits, erasable programmable read-only memory circuits, or mask read-only circuits), volatile memory circuits (such as static random access memory circuits or dynamic random access memory circuits), magnetic storage media (such as analog or digital tapes or hard disk drives), and optical storage media (such as CDs, DVDs, or Blu-ray discs).

[0061] The devices and methods described in the present application can be implemented partially or fully by a special-purpose computer created by configuring a general-purpose computer to execute one or more specific functions embodied in a computer program. The above functional blocks, flowchart components, and other elements serve as software specifications, which can be converted into a computer program by routine work of a skilled technician or programmer.

[0062] The description of the present disclosure is merely exemplary in nature, and thus, variations that do not depart from the essence of the present disclosure are intended to fall within the scope of the present disclosure. Such variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A vehicle comprising: one or more displays; The vehicle comprises a pane comprising one or more regions, wherein: a barrier is formed by the pane and separates the interior of the vehicle from the exterior of the vehicle, the barrier being one or more of transparent or translucent to all transmission of visible light from the exterior of the vehicle through the one or more regions to the interior of the vehicle, The outer perimeter of the pane forms an edge of the pane and defines a surface area of ​​the pane, and said one or more regions extend over more than 20% of said surface area; one or more processors; and a non-transitory memory comprising instructions that, when executed by the one or more processors, are operable to: A visual output is caused to adjust the transmission of visible light through the one or more regions, wherein the visual output is based on the one or more displays.

2. A vehicle as described in claim 1, wherein the one or more displays include a first display and a second display, the first display is configured to provide a first portion of the visual output based on a projector, and the second display is configured to provide a second portion of the visual output based on an organic light emitting diode integrated with the pane.

3. A vehicle as described in claim 2, wherein the one or more areas include a first area and a second area, and the first display is configured to provide the first portion of the visual output within the first area, and the second display is configured to provide the second portion of the visual output within the second area. 4 . The vehicle of claim 3 , wherein the first area forms a boundary with the second area, and a brightness of the first portion of the visual output at the boundary is less than a maximum brightness of the first portion of the visual output. 5 . The vehicle of claim 3 , wherein the first area forms a boundary with the second area, and a brightness of the first portion of the visual output at the boundary is less than a maximum brightness of the second portion of the visual output. 6 . The vehicle of claim 3 , wherein the first display includes a wedge-shaped film, and the wedge-shaped film is configured to reduce the occurrence of reflections associated with the first display or the second display.

7. The vehicle of claim 1, further comprising: A support structure for a vehicle, wherein the support structure defines an inner perimeter and the pane is sized to extend to the inner perimeter and the inner perimeter joins the outer perimeter of the pane.

8. The vehicle of claim 1, wherein the visual output is based on a digital representation and the digital representation depicts surroundings of the exterior of the vehicle.

9. The vehicle of claim 8, wherein the surroundings of the exterior of the vehicle include a rear view, and wherein the rear view is based on a perspective from a position on the vehicle.

10. The vehicle of claim 9, wherein a projector of the one or more displays is disposed on a roof of the vehicle.

11. The vehicle of claim 9, wherein the location is a side of the vehicle.

12. The vehicle of claim 1, wherein the visual output increases the opacity of the barrier.

13. The vehicle of claim 1 wherein during a first operating state, the one or more regions extend over less than 50% of the surface area, and during a second operating state, the one or more regions extend over more than 50% of the surface area.

14. The vehicle of claim 1, wherein the one or more displays include a first frit-based display, a second projector-based display, and a third organic light emitting diode-based display.

15. The vehicle of claim 1, further comprising one or more sensors configured to output one or more digital representations, wherein the one or more displays comprising a first display and a second display, the first display being configured to provide a first portion of the visual output and the second display being configured to provide a second portion of the visual output, the one or more regions include a first region and a second region, and the first display is configured to provide the first portion of the visual output within the first region, and the second display is configured to provide the second portion of the visual output within the second region, wherein the visual output comprises a first user interface element within the first portion of the visual output and a second user interface element within the second portion of the visual output, The non-transitory memory also includes a neural network and instructions that, when executed by the one or more processors, are operable to: recognizing one or more gestures, wherein said recognition of said one or more gestures is based on said one or more digital representations and said neural network, determining an operation, wherein the determination of the operation is based on the one or more gestures and the neural network, determining a selection, wherein the determination of the selection indicates the first user interface element or the second user interface element, the determination of the selection being based on the one or more gestures and the one or more neural networks, and The operations are performed, wherein the performing of the operations is based on the one or more processors and the selection.

16. The vehicle of claim 15, wherein: The one or more sensors include a first sensor positioned on a roof of the vehicle and a second sensor positioned on a dashboard of the vehicle, The one or more digital representations include a first digital representation based on the first sensor and a second digital representation based on the second sensor, and A first input to the neural network is based on the first digital representation and a second input to the neural network is based on the second digital representation.

17. A method for operating a vehicle, the method comprising: causing a visual output to adjust the transmission of visible light through a first region and a second region, wherein the visual output includes a first user interface element and a second user interface element, the first user interface element being depicted within the first region and the second user interface element being depicted within the second region; recognizing one or more gestures, wherein the recognition of the one or more gestures is based on one or more digital representations, the one or more gestures indicate an action and a selection, and the selection indicates the first user interface element or the second user interface element; as well as The operation is performed, wherein the performing of the operation is based on the selection.

18. The method of claim 17, wherein the manipulation comprises adjusting an experience provided to an occupant of the vehicle.

19. A method for training a neural network, the method comprising: managing training data, wherein the training data includes a depiction of a gesture and annotations of the gesture, and the gesture is based on a depiction of a position of a first body part and a depiction of a movement of a second body part; training the neural network based on a first subset of the training data, wherein the neural network is configured to recognize an operation and a selection of a first user interface element or a selection of a second user interface element; and The neural network is validated based on a second subset of the training data.

20. The method of claim 19, wherein the first body part comprises one or more of an eye, a head, or a neck, and the second body part comprises a hand.

Images