METHOD AND SYSTEM FOR DETERMINING WHETHER A VIEW OF A DISPLAY IS OBSTRUCTED BY A STEERING WHEEL

Abstract

According to exemplary embodiments, methods and systems are provided that include receiving, via one or more sensors of a vehicle, sensor data relating to a steering wheel of the vehicle; determining, via a processor of the vehicle using the sensor data, whether a view of a vehicle display is obscured by the steering wheel by a user of the vehicle; and adjusting, via instructions provided by the processor, one or more display images shown on the display when it is determined that the view of the display is obscured by the steering wheel by the user of the vehicle.

Description

[0001] The technical field generally relates to vehicles and in particular to methods and systems for adapting vehicle displays for optimal viewing by a vehicle user.

[0002] Many vehicles today include a display for viewing by a driver or other vehicle user, showing, for example, speed and / or other parameters related to the vehicle and / or its operation. However, such existing vehicle systems may not always provide optimal viewing of the display for the user, for example, if the user's view of one or more display parameters can be obstructed.

[0003] German patent DE 10 2014 113 308 A1 discloses an adaptive instrument display comprising a combination instrument that displays multiple indicators. A steering wheel incorporates a sensor, and a camera is also included. A processor collects data from the camera and the steering wheel sensor and determines the steering wheel position and the occupant's / driver's line of sight. A hidden section of the combination instrument is defined based on the steering wheel position and the driver's line of sight, and the instrument display indicators are modified accordingly.

[0004] German patent DE 10 2018 128 706 A1 discloses a method for controlling a privacy screen for a display unit, wherein the privacy screen protects at least part of the content displayed by the display unit. The method comprises generating a video signal; determining the respective position of one or more persons relative to the display unit based on the video signal; determining the respective viewing direction of the one or more persons; generating a prediction of the behavior of each of the one or more persons based on the detected respective position and viewing direction of each of the one or more persons, wherein the prediction includes one or more predicted viewing directions of the respective person; and controlling the privacy screen for the display unit based on the prediction.

[0005] Accordingly, it is desirable to provide improved methods and systems for adjusting vehicle displays, including for improved viewing by a vehicle user. Furthermore, other desirable features and characteristics of the present disclosure will become apparent from the following detailed description and the accompanying claims in conjunction with the accompanying drawings and the preceding technical field and background.

[0006] According to an exemplary embodiment, a method is provided which includes receiving, via one or more sensors of a vehicle, sensor data relating to a steering wheel of the vehicle; determining, via a processor of the vehicle using the sensor data, whether a view of a vehicle display is obscured by the steering wheel by a user of the vehicle; and adjusting, via instructions provided by the processor, one or more display images shown on the display when it is determined that the view of the display is obscured by the steering wheel by the user of the vehicle.

[0007] In an exemplary embodiment, obtaining sensor data also involves obtaining camera images from one or more cameras of a driver monitoring system of the vehicle; determining whether the view of the display is obscured is performed by the processor based on the camera images; and adjusting the one or more display images is performed by the processor using a machine language model.

[0008] The camera images relate to both the steering wheel and the vehicle occupant, as well as a reference point used to determine whether the user's view of the display is obstructed by the steering wheel. This determination is made by the processor based on the camera images of the steering wheel, the occupant, and the reference point. The reference point is located at the center of an airbag cover on a cabin pillar inside the vehicle.

[0009] In one exemplary embodiment, the cabin pillar inside the vehicle is also a B-pillar.

[0010] In another exemplary embodiment, the step of determining whether the view of the display is obscured is further performed by the processor using a three-dimensional coordinate representation of the camera images representing the head of a driver of the vehicle, which is used to estimate a line of sight between the driver's eyes and the display in combination with an estimated position of the steering wheel.

[0011] In an exemplary embodiment, the step of obtaining the sensor data further includes obtaining steering wheel position sensor data from one or more steering wheel position sensors of the vehicle; and determining whether the view of the display is obscured is performed by the processor based on the steering wheel position sensor data in addition to the camera images.

[0012] In an exemplary embodiment, the step of adjusting one or more display images also includes moving, via the processor, one or more display images to another section of the display that is not obscured by the steering wheel for the user.

[0013] In an exemplary embodiment, the step of adjusting one or more display images also includes rescaling, via the processor, one or more display images to a different size, so that one or more display images are no longer obscured by the steering wheel for the user.

[0014] In another exemplary embodiment, the size of the rescaling is based on whether the one or more display images represent a regulatory requirement; and a portion of the one or more display images is conditionally hidden on the display, based on whether the rescaling was successful in reducing obstruction by the steering wheel, and further based on whether the one or more display images represent a regulatory requirement.

[0015] Also in an exemplary embodiment, the machine language model includes a plurality of input layers with values ​​from the sensor data and from a machine language database, including driver eye position, steering wheel position and reference position, along with move and centralize, rescale and conditionally hidden options; a plurality of hidden layers for processing the plurality of input layers; and an output node that is generated from the plurality of hidden layers using each of the plurality of input layers, including the driver eye position, steering wheel position and reference position, along with the move and centralize, rescale and conditionally hidden options.

[0016] In another exemplary embodiment, a system is provided that includes one or more vehicle sensors and a processor. The one or more sensors are configured to receive sensor data relating to the vehicle's steering wheel. The processor is coupled to the one or more sensors and is configured to enable at least the following: determining, using the sensor data, whether a view of a vehicle display is being obscured by the steering wheel of a vehicle user; and adjusting, via instructions provided by the processor, one or more display images shown on the display when it is determined that the vehicle user's view of the display is being obscured by the steering wheel.

[0017] In an exemplary embodiment, the one or more sensors also include one or more cameras configured to receive camera images; and the processor is further configured to enable at least the following: determining whether the view of the display is obscured, based on the camera images; and adjusting the one or more display images using a machine language model.

[0018] In another exemplary embodiment, the camera images relate to both the steering wheel and the vehicle user, in addition to a reference point used to determine whether the user's view of the display is obscured by the steering wheel; and the processor is further configured to enable at least the following: determining whether the view of the display is obscured, based on the camera images of the steering wheel, the user, and the reference point.

[0019] In an exemplary embodiment, the processor is further configured to enable at least the following: Determining whether the view of the display is obscured, using a three-dimensional coordinate representation of the camera images representing the head of a driver of the vehicle, which is used to estimate a line of sight between the driver's eyes and the display in combination with an estimated position of the steering wheel.

[0020] In an exemplary embodiment, the one or more sensors further include one or more steering wheel position sensors of the vehicle configured to receive steering wheel position sensor data; and the processor is further configured to enable at least the following: determining whether the view of the display is obscured, using the steering wheel position sensor data in addition to the camera images.

[0021] In an exemplary embodiment, the processor is further configured to enable at least the following: adjusting one or more display images by moving one or more display images to another section of the display that is not obscured by the steering wheel for the user.

[0022] In an exemplary embodiment, the processor is further configured to enable at least the following: adapting one or more display images by rescaling one or more display images to a different size, so that one or more display images are no longer obscured by the steering wheel for the user.

[0023] In another exemplary embodiment, the size of the rescaling is based on whether the one or more display images represent a regulatory requirement; and a portion of the one or more display images is conditionally hidden on the display, based on whether the rescaling was successful in reducing obstruction by the steering wheel, and further based on whether the one or more display images represent a regulatory requirement.

[0024] Also in an exemplary embodiment, the machine language model includes a plurality of input layers with values ​​from the sensor data and from a machine language database, including driver eye position, steering wheel position and reference position, along with move and centralize, rescale and conditionally hidden options; a plurality of hidden layers for processing the plurality of input layers; and an output node that is generated from the plurality of hidden layers using each of the plurality of input layers, including the driver eye position, steering wheel position and reference position, along with the move and centralize, rescale and conditionally hidden options.

[0025] In another exemplary embodiment, a vehicle is provided comprising a body, a drive system configured to move the body, a steering wheel, a display, one or more sensors, and a processor. The one or more sensors are configured to receive sensor data relating to the steering wheel. The processor is coupled to the one or more sensors and is configured to enable at least the following: determining, using the sensor data, whether a user's view of the display is obscured by the steering wheel; and adjusting, via instructions provided by the processor, one or more display images shown on the display when it is determined that the user's view of the display is obscured by the steering wheel.

[0026] The present disclosure is described below in conjunction with the following drawing figures, where the same reference numerals denote the same elements and where: Fig. 1 a functional block diagram of a vehicle which includes a display, a steering wheel and a control system for controlling the display, including where a user's view of the display would otherwise be affected by the steering wheel, according to exemplary embodiments; Fig. 2. A flowchart of a process for controlling a vehicle display, including when a user's view of the display would otherwise be affected by the steering wheel, and that in conjunction with the vehicle, the display, and the control system according to Fig. 1 can be implemented according to exemplary embodiments. Fig. 3 and Fig. 4. Illustrations of an exemplary implementation of a section of the process according to Fig. 2 are, including determining whether a user's view of the display is affected by the steering wheel, according to exemplary embodiments; Fig. 5 is a representation of a machine learning model that is used in an exemplary implementation of a section of the process according to Fig. 2 is used, according to exemplary embodiments; and Fig. 6A, Fig. 6B, Fig. 7A, Fig. 7B and Fig. 8 exemplary vehicle displays as they are displayed in accordance with the process according to Fig. 2 were adapted according to exemplary embodiments.

[0027] Fig. Figure 1 illustrates a vehicle 100 according to an exemplary embodiment. As described in more detail below, the vehicle 100 includes a display 103 and a control system 102 configured to control the display 103, including when a user's view of the display 103 would otherwise be impaired, according to exemplary embodiments.

[0028] In various embodiments, Vehicle 100 includes an automobile. Vehicle 100 can be any of a number of different types of automobiles, such as a sedan, station wagon, truck, or sport utility vehicle (SUV), and in certain embodiments can be two-wheel drive (2WD) (i.e., rear-wheel drive or front-wheel drive), four-wheel drive (4WD), or all-wheel drive (AWD), and / or various other types of vehicles. In certain embodiments, Vehicle 100 can also include a motorcycle or other vehicle, such as an aircraft, spacecraft, watercraft, and so on, and / or one or more other types of mobile platforms (e.g., a robot and / or other mobile platform).

[0029] The vehicle 100 comprises a body 104 mounted on a chassis 116. The body 104 essentially encloses other components of the vehicle 100. The body 104 and the chassis 116 can together form a frame. The vehicle 100 also includes a plurality of wheels 112. The wheels 112 are each rotatably coupled to the chassis 116 near a respective corner of the body 104 to enable movement of the vehicle 100. In one embodiment, the vehicle 100 includes four wheels 112, although this may vary in other embodiments (for example, for trucks and certain other vehicles).

[0030] A drive system 110 is mounted on the chassis 116 and drives the wheels 112, for example via axles 114. In certain embodiments, the drive system 110 comprises a drive unit. In certain exemplary embodiments, the drive system 110 comprises an internal combustion engine and / or an electric motor / generator coupled to it via a transmission. In certain embodiments, the drive system 110 can vary and / or two or more drive systems 110 can be used.

[0031] As in Fig. As shown in Figure 1, the vehicle also includes a braking system 106 and a steering system 108 in various embodiments. In exemplary embodiments, the braking system 106 controls the braking of the vehicle 100 using brake components that are controlled by inputs provided by a driver (e.g., via a brake pedal in certain embodiments) and / or automatically via the control system 102. Also in exemplary embodiments, the steering system 108 controls the steering of the vehicle 100 via steering components (e.g., a steering column coupled to the axles 114 and / or the wheels 112) that are controlled by inputs provided by a driver (e.g., via the steering wheel 105 in certain embodiments) and / or automatically via the control system 102.

[0032] In various embodiments, the display 103 provides information to a driver and / or another user of the vehicle 100. For example, in various embodiments, the display 103 provides a digital cluster of display images showing parameters and information regarding the speed of the vehicle 100, along with other possible parameters relating to the vehicle 100 and / or its operation (e.g., revolutions per minute, temperature, fuel level, etc.). In various embodiments, the display 103 includes a system that incorporates a display screen for the visual presentation of this information to the user according to instructions provided by the control system 102. In certain embodiments, the display 103 may also include one or more audio, haptic, and / or other components.

[0033] In the Fig. In the embodiment shown in Figure 1, the control system 102 is coupled to the display 103. In certain embodiments, the control system 102 can also be coupled to one or more other vehicle components, such as the drive system 110, the steering system 108, the braking system 106, etc. In various embodiments, the control system 102 controls the display 103, including by adjusting images and parameters for the display 103 when a user's view of the display 103 is impaired or restricted, including when the steering wheel 105 obscures part or more of the user's view of the display 103. In various embodiments, the control system 102 performs these functions according to process 200 as described in Figure 1. Fig. 2 and the implementations of the Fig. 3-5, Fig. 6A, Fig. 6B, Fig. 7A, Fig. 7B and Fig. 8 ready, and as described in more detail below in connection therewith. In certain embodiments, the control system 102 can also control one or more other systems of the vehicle 100.

[0034] As in Fig. As shown in Figure 1, the control system 102 includes in various embodiments a sensor arrangement 120 and a controller 140, as described in more detail below.

[0035] In various embodiments, the sensor arrangement 120 includes different sensors that receive sensor data for use in determining whether a user's view of the display 130 is obstructed. In the illustrated embodiment, the sensor arrangement 120 includes one or more cameras 122 and one or more steering wheel sensors 124.

[0036] In various embodiments, the one or more cameras 122 receive camera images relating to the face of a driver or another user of the vehicle 100, and in further embodiments, also relating to the steering wheel 105 and one or more reference points of the vehicle 100, including for use in determining whether the user can view the display 130 directly without obstruction.

[0037] In addition, one or more steering wheel position sensors 124 in various embodiments detect a position of the steering wheel 105 of the vehicle 100 in the form of a steering wheel position sensor, which can also be used to determine whether the user can view the display 130 directly without obstruction.

[0038] In various embodiments, the control unit 140 is coupled to the sensor arrangement 120 and the display 103. In various embodiments, the control unit 140 can also be coupled to one or more other vehicle systems, as mentioned above. Also in various embodiments, the control unit 140 comprises a computer system (hereinafter also referred to as computer system 140) and includes a processor 142, a memory 144, an interface 146, a storage device 148, and a computer bus 150. In various embodiments, the control unit (or the computer system) 140 controls the display 103, including adjustments to it, so that the driver or another user of the vehicle 100 can view the parameters and information of the display 103 without obstruction (including without obstruction by the steering wheel 105).In various embodiments, the control unit 140 provides these and other functions according to the steps of process 200. Fig. 2 and the implementations of the Fig. 3-5, Fig. 6A, Fig. 6B, Fig. 7A, Fig. 7B and Fig. 8 ready.

[0039] In various embodiments, the controller 140 (and in certain embodiments, the control system 102 itself) is arranged within the body 104 of the vehicle 100. In one embodiment, the control system 102 is mounted on the chassis 116. In certain embodiments, the controller 140 and / or the control system 102 and / or one or more components thereof can be arranged outside the body 104, for example, on a remote server, in the cloud, or another device where the image processing is performed remotely.

[0040] It goes without saying that the control unit 140 otherwise differs from the one in Fig. The embodiment shown in Figure 1 may differ. For example, the control unit 140 may be coupled to one or more remote computer systems and / or other control systems, or may otherwise use them, for example as part of one or more of the devices and systems of the vehicle 100 identified above.

[0041] In the illustrated embodiment, the computer system of the controller 140 comprises a processor 142, a memory 144, an interface 146, a storage device 148, and a bus 150. The processor 142 performs the calculation and control functions of the controller 140 and can comprise any type of processor or multiple processors, individual integrated circuits such as a microprocessor, or any suitable number of integrated circuit devices and / or printed circuit boards working together to implement the functions of a processing unit. During operation, the processor 142 executes one or more programs 152 contained in the memory 144 and, as such, controls the general operation of the controller 140 and the computer system of the controller 140, generally when executing the processes described herein, such as process 200 according to Fig. 2 and the implementations of the Fig. 3-5, Fig. 6A, Fig. 6B, Fig. 7A, Fig. 7B and Fig. 8.

[0042] Memory 144 can be any suitable type of memory. For example, memory 144 can be various types of dynamic random access memory (DRAM), such as SDRAM, the various types of static RAM (SRAM), and the various types of non-volatile memory (PROM, EPROM, and Flash). In certain examples, memory 144 is located on the same computer chip as processor 142 and / or is located on the same computer chip as processor 142. In the illustrated embodiment, memory 144 stores the aforementioned program 152 along with stored values ​​157 (e.g., thresholds for process 200 in various embodiments).

[0043] The bus 150 serves to transmit programs, data, status, and other information or signals between the various components of the controller's computer system 140. The interface 146 enables communication with the controller's computer system 140, for example, from a system driver and / or another computer system, and can be implemented using any suitable method and device. In one embodiment, the interface 146 receives various data from the sensor arrangement 120, among other possible data sources. The interface 146 can include one or more network interfaces for communication with other systems or components.Interface 146 may also include one or more network interfaces for communication with technicians, and / or one or more storage interfaces for connection to storage devices, such as storage device 148.

[0044] The storage device 148 can be any suitable type of storage device, including various types of random-access memory and / or other storage devices. In an exemplary embodiment, the storage device 148 comprises a program product from which the memory 144 can receive a program 152 executing one or more embodiments of one or more processes of the present disclosure, such as the steps of process 200 described below in conjunction with Fig. 2 and the implementations of the Fig. 3-5, Fig. 6A, Fig. 6B, Fig. 7A, Fig. 7B and Fig. 8 will be discussed. In another exemplary embodiment, the program product can be stored directly in memory 144 and / or on a disk (e.g. disk 156), such as the one mentioned below, and / or accessed in another way.

[0045] The bus 150 can be any suitable physical or logical means for connecting computer systems and components. This includes, but is not limited to, direct hard-wired connections, fiber optic, infrared, and wireless bus technologies. During operation, the program 152 is stored in memory 144 and executed by processor 142.

[0046] It is understood that, although this exemplary embodiment is described in the context of a fully functioning computer system, those skilled in the art will recognize that the mechanisms of the present disclosure are capable of being distributed as a program product with one or more types of non-volatile, computer-readable signal-carrying media used to store the program and its instructions and to carry out its distribution, such as a non-volatile, computer-readable medium carrying the program and containing computer instructions stored therein to induce a computer processor (such as Processor 142) to execute and carry out the program. Such a program product can take a variety of forms, and the present disclosure applies equally regardless of the specific type of computer-readable signal-carrying media used to carry out the distribution.Examples of signal-carrying media include: writable media, such as floppy disks, hard drives, memory cards, and optical discs, and transmission media, such as digital and analog communication links. It is understood that cloud-based storage and / or other technologies may also be used in certain embodiments. Likewise, it is understood that the computer system of the control unit 140 may also differ from the one described in [reference to be added]. Fig. 1 can differ from the embodiment shown, for example in that the computer system of the controller 140 can be coupled with one or more remote computer systems and / or other control systems or can use them in other ways.

[0047] With further reference to Fig. 1 In certain embodiments, the control system 102 can also be designed as part of a larger system 101, which, for example, in certain embodiments also includes the display 103 and / or the steering wheel 105.

[0048] Fig. Figure 2 is a flowchart of a process 200 for controlling a vehicle display, including situations where a user's view of the display would otherwise be obstructed, according to an exemplary embodiment. In various embodiments, the process 200 can be used in conjunction with the vehicle 100 according to Fig. 1 to be implemented, including the display 103 and the control system 102 according to Fig. 1 and its components. Process 200 is also referred to below with further reference to the Fig. 3-5, Fig. 6A, Fig. 6B, Fig. 7A, Fig. 7B and Fig. 8 discusses the exemplary implementations of the process 200 according to Fig. 2.

[0049] As in Fig. As shown in Figure 2, process 200 begins at step 202 in various embodiments. In one embodiment, process 200 begins when a driving or ignition cycle of the vehicle begins, for example, when a driver enters the vehicle to operate it. In another embodiment, the steps of process 200 are performed continuously during vehicle operation.

[0050] In various embodiments, sensor data is collected while a user (e.g., driver) adjusts their position in the vehicle 100 (step 202). In various embodiments, sensor data is obtained from the sensor arrangement 120 according to Fig. 1 received, while the driver adjusts his seat together with the steering wheel 105 according to Fig. 1 adjusts before the vehicle's current journey.

[0051] In various embodiments, the steering wheel position is determined (step 204). In particular, in various embodiments, the processor 142 determines a position of the steering wheel 105 based on the sensor data according to step 202, including sensor data obtained from one or more cameras 122 and / or steering wheel sensors 124.

[0052] With reference to Fig. 3. An illustration 300 is provided relating to the determination of the steering wheel position according to step 204 in accordance with an exemplary embodiment. In particular, as shown in illustration 300, Fig. Figure 3 shows that, in an exemplary embodiment, one or more cameras of a driver monitoring system (DMS) 302 (e.g., arranged near the steering wheel 105 in an exemplary embodiment) are used to detect the current position of the steering wheel 105 relative to a known reference point 304. The reference point 304 comprises the center of an airbag cover on a pillar (such as a B-pillar) of the cabin inside the vehicle 100. Alternatively, the steering wheel position can be determined directly via sensor data obtained from one or more steering wheel sensors 124 (which, in certain embodiments, may also be part of the DMS 302).

[0053] With renewed reference to Fig. 2. In various embodiments, the position of the driver is also determined (step 206). In particular, in various embodiments, the processor 142 determines the position of the driver's eyes (and / or, in certain embodiments, the position of the head) based on the sensor data according to step 202, including sensor data obtained from one or more cameras 122.

[0054] With reference to Fig. 4. Illustration 400 is provided relating to the determination of the driver's eyes (and / or head) according to step 206 in accordance with an exemplary embodiment. In particular, as shown in Illustration 400 according to Fig. Figure 4 shows that in an exemplary embodiment, one or more cameras of the DMS 302 are used to collect camera images of the seat 406 occupied by the driver, together with the driver's head 404, in order to obtain a three-dimensional coordinate representation 402 (e.g. ΔX, ΔY, ΔZ) of the driver's eyes (or in certain embodiments, the driver's head), which is used to estimate a line of sight between the driver's eyes and the display in combination with an estimated position of the steering wheel.

[0055] With renewed reference to Fig. 2 In various embodiments, a determination is made as to whether the driver's view of the display is obstructed (step 208). In various embodiments, during step 208, the processor 142 uses the sensor data according to step 202 and the determinations of steps 206 and 208 to determine whether the driver has an obstructed view of the display 103 (e.g., of the vehicle speed, fuel range, and / or other parameters displayed on a screen of the display 103 relating to the vehicle 100 and / or its operation). For example, in certain embodiments, the determination according to step 208 includes a determination as to whether a rim, hub, or other component or section of the steering wheel 105 obscures the entire screen of the display 103 or a portion thereof, including the upper or lower boundary line of a cluster, in addition to other possible obstructions by the steering wheel 105 (e.g., the steering wheel's position).B. such that the driver's line of sight does not reach certain display images of information on the display).

[0056] If, in various embodiments, step 208 determines that the display is not obscured by the steering wheel, then the display remains unchanged (step 210). In particular, in various embodiments, if the display is not obscured, the processor 142 provides instructions to the display 103 to continue displaying the display images with parameters and information in a current or default position, size, and type of display.

[0057] Conversely, in various embodiments, if step 208 determines that the display is obscured based on the steering wheel, a machine learning database is used to determine one or more suitable adjustments for the display images (step 212). In particular, in various embodiments, the sensor data and related determinations from steps 202-208 are combined with a machine learning database (e.g.,as stored under the stored values ​​157 in the memory 144 of the vehicle 100) in conjunction with a machine learning model to determine one or more adjustments for the display of information images for the driver, so that the display (or in certain embodiments key parameters and / or other information displayed therein) is no longer obscured, but rather can be easily viewed by the driver of the vehicle 100.

[0058] With reference to Fig. Figure 5 is an exemplary model 500 for machine learning, which is used in conjunction with step 208 of process 200 according to Fig. 2 can be implemented. As in Fig. As shown in Figure 5, the machine learning model 500 in an exemplary embodiment includes a plurality of input layers 502, a plurality of hidden layers 520 and an output layer 530.

[0059] Likewise, as in Fig. As shown in Figure 5, the input layers 502, in an exemplary embodiment, include a driver's eye position 504, a steering wheel position 506, a reference position 508 (e.g., the center of the airbag cover in certain embodiments); displacement and centering options 510; rescaling options 512; and conditionally hidden options 514. In various embodiments, the driver's eye position 504, the steering wheel position 506, and the reference position 508 are determined by the processor 142 based on the sensor data and determinations in steps 204–208. Also in various embodiments, the displacement and centering options 510, the rescaling options 512, and the conditionally hidden options 514 represent possible manipulations of display images shown on the display 103, which can reduce the obscuration of the steering wheel 105 from the driver's view of the display 103.

[0060] Likewise, as in Fig. As shown in Figure 5, the hidden layers 520 in an exemplary embodiment include at least two additional layers for processing the input layers 502. In addition, the output 530 in certain embodiments provides results of the possible manipulations in view of the sensor data and other input layers 502, indicating whether and to what extent the possible manipulations can be successful in reducing or eliminating the obscuration of the steering wheel 105 from the driver's view of the display 103.

[0061] With renewed reference to Fig. 2 In an exemplary embodiment, data shifting and centralization are performed (step 214). In various embodiments, during step 214, the processor 142 applies one of the possible manipulations of the display images for the information appearing on the display 103, namely shifting and centering. In particular, in various embodiments, the processor 142 shifts display images containing information of interest (e.g., vehicle speed and other important information that the driver can see) along the display 103 to an area of ​​the display 103 that is not obscured from the driver's view by the steering wheel 105. In various embodiments, this also includes centering the display images containing information of interest in a central or middle area of ​​the section of the display 103 that is not obscured by the steering wheel 105 (e.g.,in various embodiments such that the line of sight from the driver's eyes to the display 103 is not obscured by the steering wheel 105).

[0062] In an exemplary embodiment, a determination is also made as to whether the obstruction has been removed (step 216). In certain embodiments, the processor 142 determines during step 216 whether the shifting and centering of step 214 effectively removes the obstruction, so that the driver can now successfully see the display 103 (and in particular the information of interest therein).

[0063] In various embodiments, if it is determined in step 216 that the shifting and centering of step 214 has effectively eliminated the occlusion, then the shifting is retained (step 218). In particular, in various embodiments, during step 216, the processor 142 provides instructions, implemented via the display 103, to provide information on the display to the display images, including the shifting and centering of step 216.

[0064] With reference to Fig. 6A and Fig. Illustrations 600 and 650 are provided in relation to the display according to step 218, including the displacement and centering of step 214 according to an exemplary embodiment. In particular, as shown in an exemplary embodiment, a previously hidden area 602 of a first illustration 600 is revealed. Fig. 6A moved upwards to an updated area 652, as shown in the second illustration 650 of Fig. 6B shown, which is not obscured for the driver (e.g. such that the line of sight from the driver's eyes to this area of ​​the display 103 is not obscured by the steering wheel 105).

[0065] With renewed reference to Fig. 2. If, instead, step 216 determines that the shifting and centering of step 214 has not effectively eliminated the occlusion, then process 200 in various embodiments proceeds to step 220, which is described directly below.

[0066] In various embodiments, during step 220, a determination is made as to whether obscured information on display 130 (i.e., information currently invisible or otherwise hidden from the driver) relates to regulatory versus program requirements. In various embodiments, a regulatory requirement would apply to information that is required to be visible to the driver by government and / or other regulatory authorities. Conversely, in various embodiments, a program requirement would also apply to information that is useful for the driver to view but is not required by government and / or other regulatory authorities. In various embodiments, the determinations of step 220 are made by processor 142 based on stored knowledge of regulatory and program requirements, such as in memory 144. Fig. 1 as stored values.

[0067] If, in various embodiments, during step 220, it is determined that the obscured information relates to a regulatory requirement, then the images are rescaled by a first size in the display 103 (step 222). Specifically, in various embodiments, during step 222, the processor 142 scales the display images containing the obscured information to a first rescaled size to attempt to eliminate the obscuration, while otherwise keeping the image of the information as large as possible. In one exemplary embodiment, the first rescaled size comprises a new size that is ninety percent (90%) of the original size; however, this may vary in other embodiments.

[0068] In one exemplary embodiment, a determination is also made as to whether the obscuration has been resolved (step 224). In certain embodiments, the processor 142 determines during step 224 whether the rescaling of step 222 effectively removes the obscuration, so that the driver can now successfully see the information on the display 103.

[0069] If, in various embodiments, it is determined in step 224 that the rescaling of step 224 has effectively eliminated the occlusion, then the rescaling of step 222 is retained (step 226). In particular, in various embodiments, during step 216, the processor 142 provides instructions that are implemented via the display 103 to provide the display images and information as rescaled in step 222.

[0070] With reference to Fig. 7A and Fig. 7B provides a first illustration 700 and an illustration 750 respectively relating to the display according to step 226, including the rescaling of step 222 according to an exemplary embodiment. In particular, as shown in an exemplary embodiment, a previously hidden image of a hidden area 702 of the first illustration 700 is displayed. Fig. 7A is rescaled, as in the second illustration 750 of Fig. 7B shows that the display image is no longer obscured for the driver (e.g., in such a way that the line of sight from the driver's eyes to this area of ​​the display 103 is not obstructed by the steering wheel 105). For example, as in the second illustration 750 of Fig. 7B shows the speed limit and other vehicle data rescaled so that they are no longer obscured by the steering wheel 105.

[0071] With renewed reference to Fig. 2. If, instead, step 224 determines that rescaling step 222 has not effectively eliminated the occlusion, then process 200 proceeds in various embodiments to step 228, which is described directly below.

[0072] In various embodiments, the display images are rescaled to a second size in the display 103 during step 228. In particular, in various embodiments, the processor 142 scales the obscured information to a second rescaled size during step 228 in an attempt to further eliminate the obscuration. In various embodiments, the second rescaled size of step 228 involves a further reduction in the image size, such that the second rescaled size of step 228 is smaller than the first rescaled size of step 222. In one exemplary embodiment, the second rescaled size of step 228 includes a new size such that the display image and associated information fit within the boundary of the section of the display 103 visible to the driver (e.g.,based on the line of sight from the driver's eyes to the display 103), but otherwise maximizing the size of the image of the information visible to the driver. In various embodiments, the rescaled display image from step 228 is provided to the driver via the display 103 based on instructions provided by the processor 142.

[0073] Referring again to step 220, if, instead, it is determined during step 220 that the hidden information relates to a program requirement (and not a regulatory requirement), then the images are rescaled by a third size in the display 103 (step 230). Specifically, in various embodiments, during step 230, the processor 142 scales the hidden information to a third rescaled size, such that the display images are smaller than the first rescaled size from step 222. In one exemplary embodiment, the third rescaled size from step 232 includes a new size that is seventy percent (70%) of the original size; however, this may vary in other embodiments.

[0074] In one exemplary embodiment, a determination is also made as to whether the obscuration has been resolved (step 232). In certain embodiments, the processor 142 determines during step 232 whether the rescaling of step 230 effectively removes the obscuration, so that the driver can now successfully see the information on the display 103.

[0075] If, in various embodiments, it is determined in step 232 that the rescaling of step 230 has effectively eliminated the occlusion, then the rescaling of step 230 is retained (step 234). In particular, in various embodiments, during step 234, the processor 142 provides instructions, implemented via the display 103, to provide the information as rescaled in step 230.

[0076] Conversely, if instead step 232 determines that rescaling step 230 has not effectively eliminated the occlusion, then process 200 in various embodiments proceeds to step 236, which is described directly below.

[0077] In various embodiments, the images are shifted and / or processed during step 236 so that certain information considered relatively less important (compared to other relatively more important information) is displayed. In various embodiments, this is done via the display 103 based on instructions provided by the processor 142.

[0078] With reference to Fig. An illustration 800 is provided in relation to the display according to step 236. In particular, according to an exemplary embodiment, certain information (which is considered to be relatively less important) and associated display images can be hidden from the driver's view on the display 103 to ensure that other information (such as the vehicle speed, which is considered to be relatively more important) is clearly viewed by the driver on the display 103.

[0079] Accordingly, methods, systems, and vehicles are provided for controlling a vehicle's display. In various embodiments, sensor data (including from driver monitoring cameras and / or steering wheel sensors, in various embodiments) are used to determine whether information on the display is obscured by the vehicle's steering wheel from view by a driver or other vehicle user. In various embodiments, the display images are adjusted accordingly via instructions provided by a vehicle processor to remove the obstruction so that the driver can clearly view the vehicle's display information (including ensuring a clear line of sight from the driver's eyes to the information on the display that is not obstructed by the vehicle's steering wheel).

[0080] It is understood that the systems, vehicles, and procedures may differ from those depicted in the figures and described herein. For example, vehicle 100 may differ according to Fig. 1, the control system 102 and / or the display 103 thereof and / or components thereof according to Fig. 1 vary in different embodiments. Likewise, it is understood that the steps of process 200 differ from those in Fig. 2 can be distinguished from each other and / or that different steps of the process 200 occur simultaneously and / or in a different order than that shown. Fig. 2 can take place. It is equally understood that the implementations of the Fig. 3-5, Fig. 6A, Fig. 6B, Fig. 7A, Fig. 7B and Fig. 8 can also differ in various embodiments.

Claims

[1] Procedure (200), comprising: Received (202), via one or more sensors of a vehicle (100), of sensor data relating to a steering wheel (105) of the vehicle (100); Determine (208), via a processor (142) of the vehicle (100) using the sensor data, whether a view of a display (103) of the vehicle is obscured by the steering wheel (105) of a user of the vehicle (100); and Adjusting (212), via instructions provided by the processor (142), one or more display images shown on the display (103) when it is determined that the view of the display (103) is obscured by the vehicle user (100) through the steering wheel (105), wherein the one or more sensors include one or more cameras (122) configured to receive camera images, wherein receiving (202) sensor data includes receiving camera images; wherein the camera images relate to both the steering wheel (105) and the user of the vehicle (100), in addition to a reference point (304) which is used to determine (208) whether the view of the display (103) is obscured by the steering wheel (105) by the user; wherein the determination (208) of whether the view of the display (103) is obscured is carried out by the processor (142) based on the camera images of the steering wheel (105), the user and the reference point (304), and wherein the reference point (304) comprises the center of an airbag cover on a pillar of a cabin inside the vehicle (100). [2] Method (200) according to claim 1, wherein: The receiving (202) of sensor data includes the receiving (202) of camera images via one or more cameras (122) of a driver monitoring system (302) of the vehicle (100); and the adaptation (212) of one or more display images is carried out via the processor (142) using a machine language model. [3] Method (200) according to claim 1, wherein the pillar of the cabin inside the vehicle (100) is a B-pillar. [4] Method (200) according to claim 1, wherein the step of determining (208) whether the view of the display (103) is obscured is further performed by the processor (142) using a three-dimensional coordinate representation (402) of the camera images representing a head (404) of a driver of the vehicle (100), which is used to estimate a line of sight between the driver's eyes and the display (103) in combination with an estimated position of the steering wheel (105). [5] Method (200) according to claim 2, wherein: The step of obtaining (202) the sensor data further comprises obtaining steering wheel position sensor data from one or more steering wheel position sensors of the vehicle (100); and Determining (208) whether the view of the display (103) is obscured is done by the processor (142) based on the steering wheel position sensor data in addition to the camera images. [6] Method (200) according to claim 1, wherein the step of adjusting (212) one or more display images comprises moving (214), via the processor (142), one or more display images to another section of the display (103) which is not obscured for the user by the steering wheel (105). [7] Method (200) according to claim 1, wherein the step of adapting (212) the one or more display images comprises rescaling (222) via the processor (142) the one or more display images to a different size, so that the one or more display images are no longer obscured by the steering wheel (105) for the user. [8] Method (200) according to claim 2, wherein the machine language model comprises the following: a plurality of input layers (502) with values ​​from sensor data and from a machine language database, including driver eye position (504), steering wheel position (506) and reference position (508), together with move and centralize (510), rescale (512) and conditionally hidden options (514); a plurality of hidden layers (520) for processing the plurality of input layers (502); and an output node that is generated from the plurality of hidden layers (520) using each of the plurality of input layers (502), including the driver eye position (504), the steering wheel position (506) and the reference position (508), together with the move and centralize (510), rescale (512) and conditionally hidden options (514). [9] System (101), comprising: one or more sensors of a vehicle (100), wherein the one or more sensors are configured to receive sensor data relating to a steering wheel (105) of the vehicle (100); wherein the one or more sensors include one or more cameras (122) configured to receive camera images; and a processor (142) coupled to one or more sensors and configured to enable at least the following: Determine (208), using the sensor data, whether a view of a display (103) of the vehicle (100) is obscured by the steering wheel (105) of the vehicle user (100); and Adjust (212), via instructions provided by the processor (142), one or more display images shown on the display (103) when it is determined that the view of the display (103) is obscured by the vehicle user (100) through the steering wheel (105). wherein the camera images relate to both the steering wheel (105) and the user of the vehicle (100), in addition to a reference point (304) which is used to determine (208) whether the view of the display (103) is obscured by the steering wheel (105) by the user; wherein the determination (208) of whether the view of the display (103) is obscured is carried out by the processor (142) based on the camera images of the steering wheel (105), the user and the reference point (304), and wherein the reference point (304) comprises the center of an airbag cover on a pillar of a cabin inside the vehicle (100).

Images