Vehicular display control device, vehicular display control system, vehicular display control method, and vehicular display control program

Abstract

A vehicular display control device (5x, 10) makes it possible to display camera video cooperatively across display areas of a plurality of display devices (2, 2a–2e, 2g, 2h, 2m). A display control unit (11) performs display control of camera video by setting a display area for the camera video to cooperative display across at least two of a plurality of display devices or to display at a single display device in accordance with an automatic operation level and the situation inside or outside a vehicle.

Description

Vehicle display control device, vehicle display control system, vehicle display control method, and vehicle display control program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Application No. 2024-60842 filed on April 4, 2024, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates to a vehicle display control device, a vehicle display control system, a vehicle display control method, and a vehicle display control program.

[0003] A conventional driving assistance device is disclosed in Patent Document 1. According to the driving assistance device disclosed in Patent Document 1, when a road surface recognition unit determines that the road surface of an intersection has been recognized, an image switching unit switches the image displayed on a display device that displays driving guidance for a vehicle to an image of the intersection captured by a camera.

[0004] Japanese Patent Application Laid-Open No. 2020-112927

[0005] As described in the Background Technology section, in a scene where the display image on the display device is switched to camera footage depending on the situation around the vehicle, if there is content other than camera footage that can be displayed on the display device, it is necessary to change the display mode of the camera footage to suit the situation, and there is room for further consideration in this regard.

[0006] The object of the present disclosure is to provide a vehicle display control device, a vehicle display control system, a vehicle display control method, and a vehicle display control program that are capable of appropriately displaying camera images and other displays to occupants in scenes where camera images are displayed.

[0007] According to one aspect of the present disclosure, a vehicle display control device is capable of displaying camera images by coordinating the display areas of multiple display devices. The display control unit controls the display of the camera images by setting whether the display areas of the camera images are displayed on at least two or more of the multiple display devices in a coordinated manner or on a single display device, depending on the autonomous driving level and the conditions inside or outside the vehicle. When controlling the display of the camera images on the display devices, the display control unit can change whether the camera images are displayed on multiple displays in a coordinated manner or on a single display, depending on the conditions. This allows the camera images and other displays to be displayed appropriately to occupants in various scenes where camera images are displayed.

[0008] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. 1 is an external configuration diagram of a cockpit system in the first embodiment, FIG. 2 is a diagram showing the configuration and control image of a PtoP display in the first embodiment, FIG. 3A is a block diagram showing a vehicle display control system in the first embodiment, FIG. 3B is an explanatory diagram of hardware and software structures in the first embodiment, FIG. 4 is a first flowchart showing a schematic view of processing contents in the first embodiment, FIG. 5 is a first schematic view showing a notification mode in the first embodiment, FIG. 6 is a second flowchart showing a schematic view of processing contents in the first embodiment, FIG. 7 is a third flowchart showing a schematic view of processing contents in the first embodiment, FIG. 8 is a fourth flowchart showing a schematic view of processing contents in the first embodiment, FIG. 9 is a second schematic view showing a notification mode in the first embodiment, FIG. 10 is a fifth flowchart showing a schematic view of processing contents in the first embodiment, FIG. 11 is a sixth flowchart showing a schematic view of processing contents in the first embodiment, and FIG. 19 is a flowchart No. 12 schematically showing the processing contents of the first embodiment; FIG. 20 is an explanatory diagram of the display target of the display area of ​​the display device in the first embodiment; FIG. 21 is an external configuration diagram of the cockpit system in the second embodiment; FIG. 22 is an explanatory diagram of the display target of the display device in the second embodiment; FIG. 23 is an external configuration diagram of the cockpit system in the third embodiment; and FIG. 24 is an explanatory diagram of the display target of the display device in the third embodiment.FIG. 25 is an external configuration diagram of a cockpit system in the fourth embodiment, FIG. 26 is an explanatory diagram of the display target of the display device in the fourth embodiment, and FIG. 27 is an explanatory diagram of the hardware and software structure in the fifth embodiment.

[0009] Hereinafter, several embodiments of a vehicle display control device, a vehicle display control system, a vehicle display control method, and a vehicle display control program will be described. Parts that perform the same functions in each embodiment will be assigned the same reference numerals, and descriptions thereof may be omitted.

[0010] First Embodiment A first embodiment will be described with reference to Figures 1 to 20. As shown in Figures 1 and 2, a vehicle display control system 1 is configured using a cockpit system 4 including a pillar-to-pillar display 2. Hereinafter, the pillar-to-pillar display 2 will be referred to as a PtoP display 2.

[0011] 1 and 2, the PtoP display 2 as a display device is configured with multiple displays 2a to 2c arranged side by side to form a landscape orientation. Each of the displays 2a to 2c of the PtoP display 2 is a large display configured as a liquid crystal display, an organic EL display, or the like, and is mounted on the dashboard between the left pillar PL and the right pillar PR of the vehicle. The display area of ​​the PtoP display 2 includes at least the display area of ​​the driver's seat front (hereinafter referred to as "D seat front") display 2a mounted in the area in front of the driver's seat, the display area of ​​the passenger seat front (hereinafter referred to as "P seat front") display 2c mounted in the area in front of the other passenger seats, and the display area of ​​the center information display (CID) 2b mounted in the intermediate area.

[0012] In the example shown in FIGS. 1 and 2 , the vehicle information display 2a refers to a display device installed in front of the D seat and is a display device that mainly displays vehicle information SP such as meter information. On the other hand, a P seat front display 2c is installed in front of the passenger seat (hereinafter referred to as "in front of the P seat") as a display device. The P seat front display 2c is a passenger seat display device that mainly displays information necessary for the P seat occupant, such as entertainment information, and information for the P seat occupant to enjoy entertainment in the vehicle. The center information display 2b refers to a display device installed between the D seat and the P seat and is a display device that mainly displays a menu screen that displays a selection screen for a launch application, an execution screen for the application, and information necessary for the driver and the P seat occupant. However, although modified examples will be described in the following embodiments, the number, installation conditions, and configuration of the display devices are merely examples and are not limited to the display configuration shown in this embodiment.

[0013] The PtoP display 2 is configured to be able to display various image content and text content in full graphic display on each of the displays 2a to 2c, such as meter images, images captured by the peripheral camera 23 (see FIG. 3A), entertainment information such as still images and video entertainment images, and navigation information based on navigation images including map images of the area around the current location.

[0014] As shown in FIGS. 2 and 3A, a large number of ECUs 5 are configured within the vehicle and connected to an in-vehicle network 25. The ECUs 5 are classified into a display system ECU 5x as a vehicle display control device, an in-vehicle control system ECU 5w, a periphery monitoring system ECU 5y, a driving control system ECU 5z, and a DCM 5v (or DCU) that communicates with the outside of the vehicle. DCM stands for Data Communication Module, and DCU stands for Data Communication Unit. The DCM 5v is a module for wireless communication with an external center 50 located outside the vehicle. Here, the ECUs 5 are classified as described above, but this classification method is merely an example. A certain ECU 5 may be classified across multiple categories.

[0015] The interior control system ECU 5w may be a well-known ECU 5 for door lock control, an ECU 5 for reclining control, an ECU 5 for air conditioning control, an ECU 5 for horn and lamp control, etc., and controls interior control actuators to perform various controls other than those related to driving control. For example, when the interior control system ECU 5w receives an instruction to open / close the door locks of the D seat, P seat, right rear seat, or left rear seat, it individually controls the door locks to open / close. Furthermore, when the interior control system ECU 5w receives a step-by-step instruction to recline the D seat, P seat, right rear seat, or left rear seat, it controls the reclining state of the specified seat in a step-by-step manner. Furthermore, when the interior control system ECU 5w receives an instruction to operate the horn, it turns the horn on / off, and when it receives an instruction to operate various lamps, it turns the headlights or brake lights on / off and turns the turn indicators and hazard lights on / off.

[0016] The driving control system ECU 5z may be a well-known vehicle control ECU, engine control ECU, motor control ECU, brake control ECU, steering control ECU, etc., but may also be configured as an integrated ECU that integrates all or some of these. The driving control system ECU 5z may also include an autonomous driving ECU. The autonomous driving ECU is an Autonomous Driving Electric Control Unit. The driving control system ECU 5z performs driving assistance control using well-known driving actuators and various sensors, such as an electromagnetic throttle, a brake actuator, an EPS motor, an accelerator position sensor, a brake pedal force sensor, a steering torque sensor, and a vehicle speed sensor.

[0017] When the cruise control system ECU 5z inputs an automatic control signal to the autonomous driving ECU, it activates a driving actuator (not shown) to perform a corresponding predetermined level of driving assistance or autonomous driving. For example, Level I driving assistance can perform automatic braking to avoid collisions with obstacles, following a preceding vehicle, or lane-staying to prevent the vehicle from drifting out of the lanes on either side. Level II autonomous driving can perform a combination of Level I driving assistance or an autonomous driving mode under specific conditions, such as automatically overtaking slower vehicles on a highway or automatically merging onto a highway. Note that Level II autonomous driving requires the driver to monitor the autonomous driving. At Level III or higher autonomous driving, the system performs all driving tasks while monitoring the vehicle.

[0018] Each ECU 5 is mainly composed of an SoC 30 or 31 (see below) incorporating a microcomputer equipped with various storage units 6 such as a processor, cache memory, RAM, and ROM, an I / O 15, and a bus connecting these. The storage unit 6 represents a non-transitory tangible storage medium that non-temporarily stores computer-readable programs and data. The non-transitory tangible storage medium is realized by a semiconductor memory or the like. Each ECU 5 is communicably connected to other ECUs 5 provided in the vehicle via a communication control unit 7 and an in-vehicle network 25.

[0019] In this embodiment, as shown in FIG. 2 , a plurality of ECUs 5 constituting a display-system ECU 5x constitute an HCU as an information processing device (corresponding to a vehicle display control device) 10. HCU stands for Human Machine Interface Control Unit. The display-system ECU 5x shares the processing power of its internal physical resources and controls the display on each of the displays 2a to 2c of the P-to-P display 2. For example, each ECU 5 constituting the display-system ECU 5x individually processes the display on each of the displays 2a to 2c of the P-to-P display 2. Although the in-vehicle network 25 is shown connected between the ECUs 5 constituting the display-system ECU 5x, they may also be connected by a dedicated line.

[0020] 3A, the information processing device 10 includes a control unit 11 as a display control unit, a calculation unit 12, a storage unit 6, a display processing unit 13, a sound processing unit 14, an I / O 15 that manages input or output from various devices, a communication control unit 7 that manages communication with other ECUs 5, and a wireless communication unit 27. Here, a form will be described in which output signals from major components such as a position detector 19, an operation input unit 21, an occupant monitor 22, a periphery camera 23, and a distance detection sensor 24 shown in FIG. 2 are input to the information processing device 10 via the I / O 15, but signals may also be input from other ECUs 5 that configure an in-vehicle control system ECU 5w, a periphery monitoring system ECU 5y, and a driving control system ECU 5z via an in-vehicle network 25.

[0021] Based on the control of the control unit 11, the calculation device 12 calculates the area to be displayed on the display screen of the PtoP display 2 for the content of images, sentences, characters, or symbols (hereinafter referred to as images, etc.) stored in the memory unit 6, calculates in which area of ​​the display screen of the PtoP display 2 the content of images, etc. will be displayed, and on which area the content of images, etc. will be superimposed, and outputs the content of images, etc. to the control unit 11.

[0022] The control unit 11 has a function of controlling the display of content such as images on the display screen of the P-to-P display 2 through the display processing unit 13. The display processing unit 13 processes the display of content such as images on the display screen of the P-to-P display 2 based on the control of the control unit 11. Content such as images can be displayed superimposed on each display layer on each display screen of the P-to-P display 2.

[0023] The control unit 11 also has a function as a sound notification control unit that controls the notification of sound from the speaker 18 via the sound processing unit 14. The sound processing unit 14 receives a call from the microphone 17 and outputs a call from the speaker 18 under the control of the control unit 11. When the sound processing unit 14 receives text or character content from the control unit 11, it converts the content into voice and reads it out through the speaker 18. The information processing device 10 also performs voice recognition on the voice input through the microphone 17.

[0024] The position detector 19 detects the position with high accuracy using a GNSS receiver such as a well-known GPS, and inertial sensors such as an acceleration sensor and a gyro sensor (not shown). The position detector 19 outputs a position detection signal to the control unit 11 via the I / O 15. The position identification unit 11a of the control unit 11 functions as an ADAS locator that sequentially determines the current position of the vehicle with high accuracy based on map information input from a map data input device and the position detection signal from the position detector 19. ADAS is an abbreviation for Advanced Driver Assistance Systems.

[0025] The map data input device is configured with a non-volatile memory or the like, and stores map data including various data such as link data and node data. The link data includes various data such as link identification data that identifies the links that make up roads on the map, link length data that indicates the length of the link, link orientation data that indicates the orientation of the link, link travel time data that indicates the time required to travel along the link, node coordinate data that indicates the coordinates of the nodes that make up the start and end points of the link, and link attribute data that indicates the attributes of the road. The link attribute data includes road symbols using icons that include prefecture names, city names, and information on national and prefectural roads, and character symbol information that indicates town names.

[0026] The node data includes various types of data such as node identification data that identifies nodes present on the map, node coordinate data that indicates the coordinates of the nodes, node name data that indicates the name of the node, node type data that indicates the type of node such as an intersection, connecting link data that identifies the links connecting to the nodes, etc. The map data also has associated therewith character symbol information that indicates the names of facilities such as parks using identification information such as character codes.

[0027] The display processing unit 13 can generate image content of a map of any location based on these map data, and can also generate the image content so as to include part or all of the character and symbol information described above.

[0028] The vehicle position calculated by the navigation process is expressed in a coordinate system consisting of latitude and longitude, and in this coordinate system, for example, the X axis represents longitude and the Y axis represents latitude. Note that the vehicle position can be measured based on information such as a travel distance obtained based on the sensing results of a vehicle speed sensor mounted on the vehicle, and various other configurations can be used as long as the vehicle position can be identified.

[0029] The operation input unit 21 is configured by a touch panel configured on a predetermined display among the multiple displays 2a to 2c that configure the PtoP display 2, such as the center information display 2b or the P seat front display 2c, or mechanical switches provided on the periphery of the PtoP display 2. When an operation input is made to the operation input unit 21, it is accepted via the I / O 15 and the operation input information is output to the control unit 11. The operation input unit 21 includes, for example, a power switch, a horn switch, a turn switch, a hazard lamp switch, an automatic control switch, and the like.

[0030] A vehicle occupant can issue an operation command to the control unit 11 by operating the operation input unit 21. The control unit 11 executes control based on an operation signal from the operation input unit 21. The operation input unit 21 also enables selection of content to be displayed on the PtoP display 2, operation of the air conditioning, operation of the audio, and input of navigation functions. For example, when a navigation app is running and the vehicle occupant inputs a destination into the operation input unit 21, the control unit 11 can execute navigation processing to provide guidance from the current position of the vehicle to the destination while displaying a map screen using the display processing unit 13.

[0031] When a vehicle occupant operates the operation input unit 21 while the vehicle is stopped, the occupant can instruct to move the display area of ​​the map or change the scale. When the display processing unit 13 receives an instruction through the control unit 11, it can move the display area within the image content and change the scale of the map by enlarging or reducing the display area. This allows the map to be displayed in any area, such as around the current position of the vehicle or around the destination, and the scale of the map display can also be changed.

[0032] The occupant monitor 22 functions as an occupant state detection unit that detects the state of an occupant in the vehicle. The occupant monitor 22 particularly functions as a driver state detection unit that detects the state of the driver and various operation states. The occupant monitor 22 is configured to monitor the occupant using signals output by the occupant's operation of, for example, a power switch, a horn switch, a turn switch, a hazard lamp switch, an automatic control switch, etc., and an occupant state monitor, and outputs various signals and monitor results to the control unit 11. The occupant monitor 22 may also monitor the occupant using signals from a steering sensor that detects whether the driver is gripping or steering the steering wheel, a seating sensor that detects whether the driver is seated in the seat, an accelerator pedal or brake pedal depression sensor, etc.

[0033] The power switch outputs a signal corresponding to the operation when turned on by an occupant in the vehicle cabin to start the internal combustion engine or the electric motor. The occupant status monitor includes a camera that detects the status of an occupant sitting in the D seat or P seat by capturing an image of the occupant using an image sensor and outputs an image signal. The driver's occupant status monitor is called a DSM, which stands for Driver Status Monitor.

[0034] The occupant status monitor irradiates near-infrared light onto the driver's head, acquires an image signal, analyzes the image as necessary, and outputs the image to the control unit 11. The occupant status monitor is used to detect the status of occupants such as the driver, particularly during driving assistance or autonomous driving. The turn switch is provided to activate the vehicle's turn indicators and outputs a turn signal to turn right or left in response to the occupant's operation of the turn switch. The horn switch is provided to sound the horn, or so-called alarm horn, in response to the occupant's instruction. The hazard lamp switch is provided to flash or turn off the hazard lamps in response to the occupant's instruction.

[0035] The automatic control switch is turned on by an occupant in the vehicle cabin to issue a command for automatic control of the vehicle's driving state, and outputs an automatic control signal corresponding to the operation. The control unit 11 can determine the behavior of the vehicle occupant, for example, the direction in which the eyes are facing, based on the signal from the occupant monitor 22, and can also input the operation state of the power switch, the operation state of the turn indicators and hazard lamps, command information for automatic control of the vehicle, etc.

[0036] The control unit 11 can detect the state of the driver, the passenger in the front passenger seat, or the passenger in the rear seat based on the signal monitored by the passenger monitor 22. This allows the control unit 11 to realize the function of a driver state determination unit that determines the state of the driver, and an occupant state determination unit that determines the state of each occupant, such as the passenger in the front passenger seat.

[0037] The peripheral cameras 23 constitute peripheral monitoring sensors such as a front camera that captures images in front of the vehicle, a back camera that captures images of the rear of the vehicle, corner cameras that capture images of the front and rear parts of the vehicle, side cameras that capture images of the sides of the vehicle, and electronic mirrors, which are output to the control unit 11 via the I / O 15 as image signals for the front guide monitor, back guide monitor, corner view monitor, side guide monitor, and electronic mirror, respectively, and stored in the storage unit 6. The communication control unit 7 is connected to an in-vehicle network 25 such as CAN or LIN, and controls data communication with other ECUs 5. The vehicle may also be equipped with an inter-vehicle communication system that allows data communication between the vehicle and other vehicles S, or a roadside communication system that allows data communication between the vehicle and roadside devices.

[0038] The vehicle is also equipped with a distance detection sensor 24 as a perimeter monitoring sensor that detects the distance to an obstacle. The distance detection sensor 24 is composed of a clearance sonar, LiDAR, millimeter-wave radar, or the like, and can detect vehicles, people, animals, fallen objects on the road, guardrails, curbs, trees, and the like that are in proximity to the front, front side, rear, rear, or side of the vehicle. It can also detect the direction to and distance from the obstacle. The perimeter monitoring sensor can also detect road markings on the roads surrounding the vehicle, such as lane markings, stop lines, crosswalks, road signs such as "Stop," and stop lines displayed at intersection boundaries. By using the image information from the surrounding camera 23, communication information from the vehicle-to-vehicle communication system and the road-to-vehicle communication system, and sensor information from the distance detection sensor 24, the control unit 11 can obtain the status of vehicles in the vicinity of the vehicle, for example, departure information when the preceding vehicle has started moving, approach information when a surrounding vehicle such as a following vehicle approaches the vehicle, and information on the distance between the vehicle and surrounding vehicles such as the preceding vehicle and the following vehicle.

[0039] A wireless communication unit 27 is also connected to the control unit 11 of the display system ECU 5x. The wireless communication unit 27 is capable of wirelessly connecting to a mobile terminal 28 carried by a vehicle occupant using a short-range wireless communication technology such as a wireless LAN or Bluetooth (registered trademark). The mobile terminal 28 is a so-called smartphone, tablet terminal, or mobile phone, and is equipped with a display device that displays various information and a function for inputting information. The mobile terminal 28 also has a telephone function for making calls to the outside, enabling calls to be made to the outside of the vehicle and further enabling data communication.

[0040] The wireless communication unit 27 is communicatively linked with a portable terminal 28 carried by a vehicle occupant by short-range wireless communication technology. The display system ECU 5x can communicate with the external center 50 via wireless communication and a wired communication network (not shown) from the portable terminal 28 carried by the vehicle occupant via the wireless communication unit 27. The display system ECU 5x can communicate various information with the external center 50 by wireless communication via the portable terminal 28 linked by short-range wireless communication technology.

[0041] The driver or passenger may also wear a wearable device 29 on their body. The wearable device 29 may be what is called a smart watch or smart glasses, and may be worn on the body in various forms, such as a wristwatch, eyeglasses, ring, earphones, clothing, or shoes. The wearable device 29 is operated by a microcomputer and has a short-range wireless communication function, and when communication is established with the mobile terminal 28 or the wireless communication unit 27, data can be transmitted between the wearable device 29 and the mobile terminal 28. Furthermore, the wearable device 29 has an audio input / output function, and when the driver inputs audio through the wearable device 29, the wearable device 29 can communicate with the mobile terminal 28 and make an external call. When audio is input from an external device, the wearable device 29 can output the audio to the driver.

[0042] Furthermore, the wearable device 29 is equipped with various sensors, and when worn on the body of a driver, passenger, or other occupant, it can measure vital information of each occupant. Examples of vital information that can be measured here include heart rate (pulse), body temperature, blood concentration, and blood pressure. Other information that can be measured include the number of steps taken, exercise records, calories burned, and sleep time. Because the wearable device 29 is always worn on the body, vital information can be constantly obtained even if the driver experiences some kind of abnormality while driving or takes a break from driving during automated driving. Furthermore, even if a passenger falls asleep during manual driving or if the driver falls asleep during automated driving, the wearable device 29 can detect this sleep information.

[0043] 3B shows an example of the hardware and software configuration of the information processing device 10. Each of the ECUs 5, 5a constituting the information processing device 10 is equipped with a SoC 30, 31, respectively, and the above-mentioned microcomputer is incorporated into the SoC 30, 31. The microcomputer incorporated into the SoC 30, 31 of the ECU 5 is configured to run a variety of applications 33 (hereinafter abbreviated as apps 33) on a pre-installed general-purpose OS 32, for example, Linux OS (Linux is a registered trademark). SoC stands for System On Chip.

[0044] The applications 33 include an image processing application 34, a navigation application, and other applications. In response to a drawing request from the image processing application 34, the SoC 30 performs drawing processing on the display screen of each display 2a of the PtoP display 2.

[0045] On the other hand, ECU 5a is provided for meter drawing purposes and is therefore designated by the reference numeral 5a. A real-time OS 35, which has higher real-time performance than the general-purpose OS 32, is installed in the microcomputer built into SoC 31 of ECU 5a, and meter application 36 is configured to run on real-time OS 35. Note that although various functions are realized by hardware executing software, the following explanation may focus on the software of application 33, such as image processing application 34 and meter application 36, for ease of understanding.

[0046] The meter application 36 is software for notifying occupants, particularly the driver, of the vehicle speed, engine speed, warnings, etc., and draws meter image content in a specific display area of ​​the PtoP display 2. For example, the meter application 36 draws image content such as a speedometer, a tachometer, a shift range position status, or warning lights. The speedometer includes a speed image whose display needs to be updated in real time to show changes in vehicle speed. Similarly, the tachometer is also included in the meter image because its display needs to be updated in real time to show changes in engine speed.

[0047] The content drawn by the meter application 36 requires relatively higher real-time performance than the content drawn by other applications. The applications 33 include a navigation application. The navigation application realizes the navigation function described above and also draws image content such as a navigation screen including a map and the current vehicle position on the PtoP display 2.

[0048] The applications 33 also include an image synthesis application. The image synthesis application specifies the size and type of various image contents to be displayed on the PtoP display 2, synthesizes parts of the image contents within one frame, and outputs this synthesized mixed image to the PtoP display 2. The image synthesis application realizes the function of an image synthesis unit, also called a compositor, and the function of an image output unit.

[0049] A display layer for drawing the image content is assigned to each of the applications 33 and 36 that draw image content. These display layers are allocated in the storage unit 6 in a size that allows the necessary image content to be drawn.

[0050] Furthermore, the image content displayed on the PtoP display 2 can be animated. Here, animation refers to a display mode in which the position or size of the image showing the content gradually changes, the image rotates, the entire user interface moves in response to a swipe operation, the image gradually fades in or out, or the color of the image changes. The control unit 11 executes the app 33 based on the navigation app and performs various processes to function as the position identification unit 11a. The control unit 11 executes the app 33 and performs various processes to function as the autonomous driving level determination unit 11c, the use determination unit 11d, the vicinity determination unit 11e, and the tire display determination unit 11f according to the present application.

[0051] The autonomous driving level determination unit 11c has a function of determining whether the autonomous driving level is autonomous driving or manual driving. The usage determination unit 11d has a function of determining whether the P seat front display 2c is being used. The P seat front display 2c corresponds to a display device for the passenger seat. The surroundings determination unit 11e has a function of determining whether a pedestrian T, another vehicle S, or a foreign object U is present in the surroundings. The tire display determination unit 11f has a function of determining whether a tire Ta should be displayed.

[0052] This embodiment is characterized by the display control content when camera images are displayed on the multiple displays 2a to 2c. Specific processing will be described below with reference to FIG.

[0053] The control unit 11 controls the display of the camera images by setting whether to display the camera images from the peripheral camera 23 in at least two or more of the display areas of the multiple displays 2 a to 2 c in a coordinated manner, or to display them in the display area of ​​a single display 2 a, 2 b, or 2 c, depending on the various input information. When the coordinated display is set, the control unit 11 controls the display of the camera images in a coordinated manner across the display areas of the multiple displays 2 a to 2 c.

[0054] A specific example is shown in Figure 4. While the host vehicle is traveling or stopped, the peripheral camera 23 captures camera video in S11. When the camera video is captured, the information processing device 10 stores the camera video in the storage unit 6. The control unit 11 performs image analysis of the camera video stored in the storage unit 6 to determine whether an intersection with poor visibility is displayed in S12. The control unit 11 may determine whether an intersection with poor visibility is displayed by adding current location information acquired by executing a navigation application to the image analysis results of the camera video.

[0055] For example, if the camera image does not show an intersection with poor visibility, the control unit 11 exits the subroutine shown in Figure 4, but if it determines in S12 that an intersection with poor visibility is shown, it determines in S13 whether the P seat front display 2c is being used. For example, the control unit 11 determines whether entertainment information or navigation information is already being displayed on the P seat front display 2c.

[0056] The control unit 11 sets whether to output a coordinated display to multiple displays including the P seat front display 2c (for example, the entire PtoP display 2) or to display on a single display (for example, the center information display 2b) depending on whether the P seat front display 2c is being used.

[0057] For example, if the control unit 11 is already controlling the display of other content on the P seat front display 2c, it determines YES in S13, and controls the display of the camera image on the single center information display 2b in S14, while leaving the content already displayed on the P seat front display 2c as it is.

[0058] Conversely, if the P seat front display 2c is not being used, the control unit 11 determines NO in S13 and controls the coordinated display to cover the entire surface of multiple displays including the P seat front display 2c, for example, the P to P display 2. For example, the control unit 11 sets the camera image to be output in wide coordinated format on the entire surface of the P to P display 2. As a result, the display processing unit 13 processes the coordinated display in wide format to cover the entire surfaces of the D seat front display 2a, center information display 2b, and P seat front display 2c that make up the P to P display 2.

[0059] <When the vehicle approaches an intersection with poor visibility> A display example is shown in Fig. 5. For example, when the vehicle approaches an intersection with poor visibility such as the T-junction shown in Fig. 5, other vehicles S may be stopped or moving slowly, or pedestrians T may be walking, on the roads passing on either side of the T-junction ahead.

[0060] In this case, the display processing unit 13 composites and outputs the camera images from the peripheral cameras 23 for wide coordinated display under the control of the control unit 11. In the example shown in Fig. 5, the control unit 11 issues a command to composite the camera images from the front camera and the corner cameras capturing the front side of the vehicle, among the camera images from the peripheral cameras 23 sequentially stored in the storage unit 6, and the display processing unit 13 outputs the composite images to each of the displays 2a to 2c. The control unit 11 controls the wide coordinated display to cover the entire surface of each of the displays 2a to 2c constituting the PtoP display 2, allowing the driver and passengers to get a bird's-eye view of the entire road with poor visibility, making it easier to notice other vehicles S and pedestrians T.

[0061] In the cases shown in Figures 4 and 5, the control unit 11 controls the display of the camera images by setting whether to display the camera images from the peripheral camera 23 on at least two or more displays in a coordinated manner or on a single display, depending on information about the situation outside the vehicle. This makes it possible to set whether to display the images on each display in a coordinated manner or on a single display depending on the situation outside the vehicle, allowing occupants to visually confirm information that is most important to them. The control unit 11 also controls the display of the camera images by setting the coordinated display on displays 2a to 2c depending on the determination result of whether the P seat front display 2c is being used. This makes it possible to set whether to display the images on each display in a coordinated manner or on a single display depending on the usage status of the P seat front display 2c.

[0062] Furthermore, as illustrated in Figure 5, the display areas of the electronic mirror's display windows 2x and 2y may be arranged to be superimposed on the D seat front display 2a and the P seat front display 2c, respectively, and the control unit 11 may control the display of the camera image captured by the corner camera of the peripheral camera 23 of the rear side inside the display windows 2x and 2y.

[0063] Another example of processing operation is shown in Fig. 6. First, while the host vehicle is traveling or stopped, the peripheral camera 23 captures camera video in S11 of Fig. 6. When the camera video is captured, the information processing device 10 stores the camera video in the storage unit 6. The control unit 11 performs image analysis of the camera video stored in the storage unit 6 to determine whether an intersection with poor visibility is displayed in S12. The control unit 11 may determine whether an intersection with poor visibility is displayed by taking into account current location information acquired by executing a navigation application in the image analysis result of the camera video.

[0064] For example, if the camera image does not show an intersection with poor visibility, the control unit 11 exits the subroutine shown in Figure 6, but if it determines in S12 that an intersection with poor visibility is shown, it determines in S23 whether or not the vehicle is in automatic driving mode.

[0065] For example, if the autonomous driving level is level zero, I, or II, the control unit 11 determines in S23 that the vehicle is being driven manually, including assistive driving, and in S24 controls the camera image to be displayed in wide format by cooperatively outputting the camera image to the entire surface of the PtoP display 2. Conversely, if the control unit 11 determines that the vehicle is being driven autonomously at level III or higher, it controls the camera image to be displayed in the display area of ​​the center information display 2b in the middle in S25.

[0066] In the case shown in Figure 6, the control unit 11 controls the display of the camera images by setting whether to display the images on the displays 2a to 2c in a coordinated manner or on a single display (for example, 2b) depending on the determination result of whether the vehicle is in autonomous driving. This makes it possible to set whether to display the images on each display in a coordinated manner or on a single display depending on whether the vehicle is in autonomous driving. Whether to display the camera images in a coordinated manner or on a single display was determined depending on whether the autonomous driving level is level II or lower or level III or higher, but this is not limiting and the determination may also be based on whether the autonomous driving level is level I or lower or level II or higher.

[0067] Another example of processing operation is shown in Fig. 7. First, while the host vehicle is traveling or stopped, the peripheral camera 23 captures camera video in S11 of Fig. 7. When the camera video is captured, the information processing device 10 stores the camera video in the storage unit 6. The control unit 11 performs image analysis of the camera video stored in the storage unit 6 to determine whether an intersection with poor visibility is displayed in S12. The control unit 11 may determine whether an intersection with poor visibility is displayed by taking into account current location information acquired by executing a navigation application in the image analysis result of the camera video.

[0068] For example, if the camera image does not show an intersection with poor visibility, the control unit 11 exits the subroutine shown in Figure 7, but if it determines in S12 that an intersection with poor visibility is shown, it determines in S33 whether a pedestrian T or another vehicle S is present.

[0069] For example, if the control unit 11 determines through image analysis that a pedestrian T or another vehicle S is present, the control unit 11 displays and outputs the camera image in a manner that calls attention to the pedestrian T or the other vehicle S by encircling them with a frame as shown in Fig. 5 in S34. Conversely, the control unit 11 may control the display of the camera image on a part of the center information display 2b in the middle in S35.

[0070] 7, the control unit 11 controls the display of the camera images by setting whether to display the images on the displays 2a to 2c in a coordinated manner or on the single center information display 2b, depending on the result of the determination by the periphery determination unit 11e as to whether a pedestrian T or another vehicle S is present. This makes it possible to set whether to display the images on the displays in a coordinated manner or on a single display, depending on the situation around the vehicle.

[0071] <When driving or driving slowly on a narrow road> Fig. 8 also shows modified display modes depending on the location. First, while the host vehicle is driving or stopped, the peripheral camera 23 captures camera video in S11 of Fig. 8. When the camera video is captured, the information processing device 10 stores the camera video in the storage unit 6. The control unit 11 performs image analysis of the camera video stored in the storage unit 6 to determine whether the vehicle is driving on a narrow road in S42. The control unit 11 may determine whether the vehicle is driving on a narrow road by taking into account current location information acquired by executing a navigation app and the image analysis result of the camera video.

[0072] For example, if the control unit 11 determines that the vehicle is traveling on a narrow road, it exits the subroutine shown in Figure 8, but if it determines in S42 that the vehicle is traveling on a narrow road, it determines in S43 whether the P seat front display 2c is being used.

[0073] For example, if the control unit 11 is already controlling the display of other content on the P seat front display 2c, it determines YES in S43, and controls the display of the camera image on the single center information display 2b in S44, while leaving the content already displayed on the P seat front display 2c as it is.

[0074] Conversely, for example, if the P seat front display 2c is not being used, the control unit 11 determines NO in S43 and controls the coordinated display on multiple displays including the P seat front display 2c, for example, the entire surface of the P to P display 2. That is, the control unit 11 controls the wide coordinated display of the camera image on the entire surface of the P to P display 2. As a result, the display processing unit 13 performs coordinated display processing in wide format on the entire surfaces of the D seat front display 2a, center information display 2b, and P seat front display 2c that make up the P to P display 2. In this case, the same effects as those of the processing operation shown in FIG. 4 can be obtained.

[0075] When the vehicle is moving slowly on a narrow road, the control unit 11 may control the display of one of the following: camera images of the front taken by the front camera, camera images under the hood of the vehicle, camera images of the side of the vehicle taken by the side camera, or a bird's-eye view created by combining multiple camera images to look down on the vehicle from above, either in conjunction with one another on multiple displays, for example, the entire surface of the PtoP display 2, or on a single display, for example, the center information display 2b.

[0076] For example, as shown in Fig. 9, while the vehicle is traveling slowly on a narrow road, the control unit 11 may perform image analysis of the camera image to discover a foreign object U in the camera image showing the area under the hood. This foreign object U may be, for example, a small child playing by hiding under the vehicle, a small animal hiding under the vehicle, a step that marks the boundary between the roadway and the sidewalk, or a parking stop block. As in the example of Fig. 9, the control unit 11 displays the camera image under the hood on the entire surface of the PtoP display 2 via the display processing unit 13, making it easier for the driver and passengers to notice the foreign object U.

[0077] 9, the control unit 11 controls the display of the camera images by setting whether to display the images on the entire P-to-P display 2 in a coordinated manner or on a single display, such as the center information display 2b, depending on the result of the determination by the periphery determination unit 11e as to whether or not a foreign object U is present. This makes it possible to set whether to display the images on each display in a coordinated manner or on a single display depending on the situation around the vehicle.

[0078] Furthermore, even when the control unit 11 controls the display of the camera image on the D-seat front display 2a through the display processing unit 13, it is required by law to control the display of vehicle information SP, including vehicle speed. When the control unit 11 displays the camera image on the entire surface of the PtoP display 2, it is preferable to superimpose the vehicle information SP on the camera image. In this case, it is preferable that the display processing unit 13 superimposes information different from the camera image (here, the vehicle information SP) on an upper layer of the display area displaying the camera image, and further controls the display so that the other information is transparent so that the camera image can also be seen.

[0079] Furthermore, when the control unit 11 controls the display of other information from the upper layer of the camera image, it is desirable that the control unit 11 display the other information in a display area where an image of low importance is displayed in the camera image. The display area of ​​the image of low importance here is preferably an area of ​​the entire camera image where an object of high importance is not detected by image recognition. Examples of objects of high importance include the foreign object U, other vehicle S, pedestrian T, tire Ta, etc., as described above.

[0080] <Changing the display mode according to tire Ta display settings> Furthermore, when the control unit 11 controls the display of the left and right tires Ta using camera images via the display processing unit 13, the driver can get a sense of the vehicle width by checking the left and right tires Ta and comparing the vehicle width with the road width. For this reason, it is desirable for the control unit 11 to unconditionally display the left and right tires Ta and the camera images of their surroundings on the entire surface of the PtoP display 2. Even if there is a fence or a ditch beside a narrow road, it becomes easier to drive safely by avoiding contact with the fence or running off into the ditch.

[0081] The control unit 11 may allow a user to select and input, by operating the operation input unit 21, whether to display the camera image from the front camera showing the tire Ta or not to display the tire Ta in conjunction with the image. When a selection is made to display the tire Ta, the control unit 11 controls the display of the camera image as is. When a selection is made not to display the tire Ta, the control unit 11 may control the display by cutting out an image of the tire Ta from the camera image or by using an image erasing technique to partially erase the tire Ta and blend it into the background. Alternatively, the control unit 11 may control the display by using an image captured by a peripheral camera 23 that is installed so as not to capture the tire Ta.

[0082] By driving while displaying tire Ta, a driver who is not accustomed to driving can avoid contact with obstacles and drive safely as described above, while a driver who is accustomed to driving can drive based on his or her own intuition by not displaying tire Ta.

[0083] An example of processing is shown in Fig. 10. While the host vehicle is traveling or stopped, in S11 of Fig. 10, the peripheral camera 23 captures camera images and stores the camera images in the storage unit 6. The control unit 11 performs image analysis of the camera images stored in the storage unit 6, and determines whether the host vehicle is traveling on a narrow road in S42. The control unit 11 may determine whether the host vehicle is traveling on a narrow road by adding current location information acquired by executing a navigation application to the image analysis results of the camera images.

[0084] For example, if the control unit 11 determines that the vehicle is traveling on a narrow road, it exits the subroutine shown in Figure 10, but if it determines in S42 that the vehicle is traveling on a narrow road, it determines in S46 whether the tire Ta is set to be superimposed.

[0085] If the superimposed display setting for tire Ta is enabled, the control unit 11 determines YES in S46, and in S47 displays the tire Ta in a wide, linked manner across the entire surface of the PtoP display 2. Conversely, if the superimposed display setting for tire Ta is not enabled, in S48 the control unit 11 controls the display of the camera image without displaying tire Ta on a single display, for example, the center information display 2b.

[0086] As a result, the control unit 11 can control the display of the camera images by setting whether to display the images on the multiple displays 2a to 2c in a coordinated manner or on a single display, such as the center information display 2b, depending on the determination result of the tire display determination unit 11f. This allows the display mode to be changed depending on the determination result of whether to display the tire Ta.

[0087] When the linked display is performed on the entire surface of the P-to-P display 2, the control unit 11 may control the display of the camera image showing the rear side on the P-seat front display 2c using the wipe display windows 2x, 2y, etc. When the linked display is performed on the entire surface of the P-to-P display 2, navigation information such as a map may be wipe-displayed on the P-seat front display 2c.

[0088] <Changing the display mode according to the autonomous driving level> In addition, the control unit 11 may control the display of the camera image by setting whether to display it in conjunction with the displays 2a to 2c or to display it on a single display, for example, the center information display 2b, depending on the result of the autonomous driving level determination by the autonomous driving level determination unit 11c.

[0089] An example of processing is shown in Fig. 11. While the host vehicle is traveling or stopped, in S11 of Fig. 11, the peripheral camera 23 captures camera images and stores the camera images in the storage unit 6. The control unit 11 performs image analysis of the camera images stored in the storage unit 6, thereby determining in S42 whether the host vehicle is traveling on a narrow road. The control unit 11 may determine whether the host vehicle is traveling on a narrow road by adding current location information acquired by executing a navigation application to the image analysis results of the camera images.

[0090] If the control unit 11 determines that the vehicle is traveling on a narrow road, it exits the subroutine shown in FIG. 11 . However, if it determines in S42 that the vehicle is traveling on a narrow road, it determines in S49 whether the vehicle is traveling autonomously. The "autonomous driving" referred to here does not only refer to an autonomous driving level such as Level III, where the driver does not need to check the road ahead. It also includes cases where the vehicle autonomously drives or steers autonomously, even though the driver is responsible for driving, such as Level II. From the perspective of driving technology, "autonomous driving" includes cases where the driving control system ECU 5z automatically controls only the drive function of the driving wheels or only the steering function in the autonomous driving function.

[0091] When considering the autonomous driving level, if the level is zero or I, the control unit 11 determines that the vehicle is being driven manually, including driving assistance, in S49, and displays a wide, linked image across the entire PtoP display 2 in S50. If the driver operates either the accelerator or the steering wheel, or both, the vehicle is being driven manually, and a wide, linked image is displayed. At this time, the content displayed on the P-seat front display 2c, such as navigation information, may be wiped or erased. Conversely, if the autonomous driving level is Level II or higher, the control unit 11 controls the camera image to be displayed on a single display, such as the center information display 2b, without a linked image in S48. This is because, in the case of autonomous driving at Level II or higher, the driver does not need to operate the accelerator or steering wheel, and therefore there is less need for a wide, linked image.

[0092] Specifically, from the perspective of the autonomous driving level, for example, when driving on a narrow road at level 0 or I, the driver is required to perform at least some driving operations, so it is advisable to control the displays 2a to 2c to display images in a coordinated manner to make it easier to recognize the actual road width. If the autonomous driving level is level II or higher, the driving control system ECU 5z executes control for autonomous driving, so the control unit 11 controls the camera images around the vehicle to be displayed on a single center information display 2b rather than on the displays 2a to 2b in a coordinated manner. In this case, the control unit 11 may control the display of the tire Ta captured by the peripheral camera 23 or auxiliary lines to a parking space. While the determination of whether to display the camera images in a coordinated manner or on a single display is based on whether the autonomous driving level is level I or lower or level II or higher, this is not limiting. The determination may also be based on whether the autonomous driving level is level 0 or higher or level I or higher, or whether the autonomous driving level is level II or lower or level III or higher.

[0093] <Changing the display mode when backing into a parking space> The control unit 11 may also control the display of the camera images by setting whether to display the images in coordination on each of the displays 2a to 2c or on a single display, such as the center information display 2b, depending on whether the P seat front display 2c is being used when backing into a parking space.

[0094] An example of processing is shown in Fig. 12. While the host vehicle is traveling or stopped, in S11 of Fig. 12, the peripheral camera 23 captures camera images and stores the camera images in the storage unit 6. In S42a, the control unit 11 detects whether the host vehicle is being reversed into parking by using a shift position sensor or the like.

[0095] When the control unit 11 determines that the vehicle is backing into a parking space, it determines in S43 whether the P seat front display 2c is being used. If the P seat front display 2c is being used, the control unit 11 outputs the camera image to a single center information display 2b while leaving the image on the P seat front display 2c. Conversely, when the P seat front display 2c is not being used, the control unit 11 controls the display by outputting the camera image in a wide coordinated manner across the entire P-to-P display 2. This makes it easier for the driver to view and understand the camera image from the backup camera while backing into a parking space. Figure 13 shows an example of a wide coordinated display while backing into a parking space. Because the rear of the vehicle is displayed across the entire P-to-P display 2, the driver can easily grasp the distance to another vehicle S parked adjacently, making parking easier.

[0096] Another processing example is shown in Fig. 14. While the host vehicle is traveling or stopped, in S11 of Fig. 13, the peripheral camera 23 captures camera images and stores the camera images in the storage unit 6. In S42a, the control unit 11 detects whether the host vehicle is being reversed into parking by using a shift position sensor or the like.

[0097] When the control unit 11 determines that the vehicle is being reversed into a parking space, it determines in S51 whether a foreign object U or another vehicle S is present. If it determines that neither a foreign object U nor another vehicle S is present, it outputs the camera image to, for example, a single center information display 2b without any linked display in S48. Conversely, if a foreign object U or another vehicle S is present, it controls the display of the camera image by linking and outputting it in wide format across the entire P-to-P display 2 in S50. This makes it easier for the driver to view and understand the camera image from the backup camera while backing into a parking space. As shown in FIG. 13 , because the rear of the vehicle is displayed across the entire P-to-P display 2, the driver can easily grasp the distance to the other vehicle S parked adjacently, making parking easier.

[0098] Another processing example is shown in Fig. 15. While the host vehicle is traveling or stopped, in S11 of Fig. 15, the peripheral camera 23 captures camera images and stores the camera images in the storage unit 6. In S42a, the control unit 11 detects whether the host vehicle is being reversed into parking by using a shift position sensor or the like.

[0099] When the control unit 11 determines that the vehicle is being reversed into parking, it determines in S52 whether the superimposed display of tire Ta is set. If the superimposed display of tire Ta is set, the control unit 11 controls the display of an image in which tire Ta is superimposed by outputting the camera image in wide coordinated display on the entire surface of the PtoP display 2 in S50. Conversely, if the superimposed display of tire Ta is not set, the control unit 11 controls the display of the camera image without coordinated display, for example, by outputting the camera image to a single center information display 2b without superimposed display of tire Ta in S48. This makes it possible to change the display mode of the displays 2a to 2c depending on whether tire Ta is to be displayed.

[0100] Another processing example is shown in Fig. 16. While the host vehicle is traveling or stopped, in S11 of Fig. 16, the peripheral camera 23 captures camera images and stores the camera images in the storage unit 6. In S42a, the control unit 11 detects whether the host vehicle is being reversed into parking by using a shift position sensor or the like.

[0101] If the control unit 11 determines that the vehicle is backing into a parking space, it determines in S53 whether driving assistance for automatic parking is being performed. If driving assistance for automatic parking is being performed, the control unit 11 controls the display of the image by outputting the camera image in wide coordinated format on the entire surface of the PtoP display 2 in S50. Conversely, if driving assistance for automatic parking is not being performed, the control unit 11 outputs the camera image to, for example, a single center information display 2b without coordinated display in S48. This makes it possible to change the display mode of the displays 2a to 2c depending on whether driving assistance for automatic parking is being performed.

[0102] <Changing Display Mode When Foreign Object U is Detected on Travel Route> Another processing example is shown in Fig. 17. While the host vehicle is traveling or stopped, in S11 of Fig. 17, the peripheral camera 23 captures camera images and stores the camera images in the memory unit 6. The control unit 11 performs image analysis of the camera images stored in the memory unit 6, and determines in S42b whether or not a foreign object U can be detected on the travel route.

[0103] When the control unit 11 detects a foreign object U, it determines in S43 whether the P-seat front display 2c is in use. If the P-seat front display 2c is not in use, the control unit 11 controls the display of the image by outputting the camera image in wide coordinated format on the entire P-to-P display 2 in S45. Conversely, if the P-seat front display 2c is in use, the control unit 11 controls the camera image to be output to, for example, a single center information display 2b without coordinated display in S44. This allows the display mode of the displays 2a to 2c to be changed depending on whether or not a foreign object U is present on the driving route. As illustrated in FIG. 18 , when the control unit 11 detects a foreign object U under the hood in the image captured by the peripheral camera 23, it is preferable to control the camera image captured under the hood to be displayed on the entire P-to-P display 2, provided that the P-seat front display 2c is not in use.

[0104] Another processing example is shown in Fig. 19. While the host vehicle is traveling or stopped, in S11 of Fig. 19, the peripheral camera 23 captures camera images and stores the camera images in the memory unit 6. The control unit 11 performs image analysis of the camera images stored in the memory unit 6, and determines in S42b whether or not a foreign object U can be detected on the traveling route.

[0105] When the control unit 11 detects a foreign object U, it determines in S46 whether the superimposed display of tire Ta is set. If the superimposed display of tire Ta is set, the control unit 11 controls the display of an image in which tire Ta is superimposed by outputting the camera image in wide coordinated display on the entire surface of the PtoP display 2 in S47. Conversely, if the superimposed display of tire Ta is not set, the control unit 11 controls the display of the camera image in S48 without coordinated display, that is, outputting the camera image without superimposed display of tire Ta on a single center information display 2b, for example. This makes it possible to change the display mode of the displays 2a to 2c depending on whether tire Ta is to be displayed. If the superimposed display of tire Ta is set on the entire surface of the PtoP display 2, the control unit 11 may control the display of a camera image showing the rear side of the P seat on the P seat front display 2c by using a wipe or the like.

[0106] The control unit 11 may allow the user to select and input, by operating the operation input unit 21, whether to display the camera image from the front camera showing the tire Ta or not to display the tire Ta in conjunction with the image. When a selection is made to display the tire Ta, the control unit 11 controls the display of the camera image as is. When a selection is made not to display the tire Ta, the control unit 11 may control the display by cutting out an image of the tire Ta from the camera image, or by using a partial image erasure technique to partially erase the tire Ta and blend it into the background. Alternatively, the control unit 11 may control the display by using an image captured by a peripheral camera 23 that is installed so as not to capture the tire Ta. This allows the display to be comfortable and easy to understand for the occupant.

[0107] Although specific scenes have been given as examples in the above, the present invention is not limited to the above-described scenes. For example, even if the control unit 11 detects an object such as a foreign object U, a pedestrian T, or another vehicle S in another driving scene, the control unit 11 may control the display to include the detected object such as the foreign object U, the pedestrian T, or the other vehicle S in the camera image. In other words, when caution is required or when there is a risk, it is preferable to display the camera image including the detected object. When it is determined that there is a particular risk, it is preferable to control the display to cover the entire surface of the PtoP display 2.

[0108] <Changing the Display Mode According to the Situation Inside the Vehicle> The control unit 11 may control the display of the camera image by setting whether to display the display area of ​​the camera image on at least two or more of the display areas of the multiple displays 2a to 2c in a coordinated manner, or to display the camera image on a single display, such as the center information display 2b, depending on the situation inside the vehicle. For example, the control unit 11 may determine whether the passenger in the P seat is watching the P seat front display 2c from the monitor signal of the occupant monitor 22, and, depending on this determination result, select whether to display the camera image on the multiple displays 2a to 2c or on a single display (e.g., 2b), and control the display of the camera image. When the control unit 11 determines that the passenger in the P seat is watching the P seat front display 2c, the control unit 11 may control the display of the camera image to be displayed on the D seat front display 2a (excluding the P seat front display 2c) and the center information display 2b in a coordinated manner, or to display the camera image on the single center information display 2b. Conversely, when the control unit 11 determines that the passenger in the P seat is not viewing the P seat front display 2c, it may control the camera image to be displayed in wide format on at least the P seat front display 2c and the center information display 2b. It may also be controlled to display the camera image in wide format on the entire P-to-P display 2.

[0109] Additionally, for example, while the vehicle is driving at autonomous driving level III or higher, the driver may watch some entertainment content, check destination information or email, or even hold a remote conference. When the driver uses private content in this manner, the control unit 11 may control the display of only the D-seat front display 2a. When the control unit 11 detects that the vehicle is experiencing the above-described conditions based on the operation input information from the operation input unit 21, the sensor signal from the passenger monitor 22, or the content display content, it may control the display of various content only on the D-seat front display 2a in front of the driver. In the case of manual driving or manual driving, a wide variety of driving-related information is displayed, so there is a high need for coordinated display of wide camera images across multiple displays (e.g., 2a-2c). Conversely, in the case of autonomous driving or automatic driving, the amount of driving-related information to be displayed is reduced, so the priority of displaying camera images across multiple displays (e.g., 2a-2c) is lowered. For this reason, if some content is being controlled to be displayed on some displays (for example, the display 2c in front of the P seats) and is already in use, it is advisable to avoid full-screen display on the PtoP display 2 by controlling the display to be displayed on other displays (for example, the display 2a in front of the D seats and the center information display 2b).

[0110] <Description of Display Target> In the above, when a single display is set as the display control target, the center information display 2b is primarily set as the display target, as shown by the hatching in the upper part of Fig. 20. Alternatively, when the control unit 11 controls the display of a single display as the display target, the display 2c in front of the P seat or the display 2a in front of the D seat may be set as the display target. Furthermore, when the control unit 11 performs linked display on multiple displays, it is desirable to set the entire surface of the PtoP display 2 as the display target, as shown by the hatching in the lower part of Fig. 20.

[0111] Additionally, when displaying information in a coordinated manner on multiple displays, only the center information display 2b and the P seat front display 2c may be the targets of the coordinated display. In other words, the D seat front display 2a does not have to be the target of the coordinated display. Also, only the D seat front display 2a and the center information display 2b may be the targets of the coordinated display. In other words, the P seat front display 2c does not have to be the target of the coordinated display. Also, only the D seat front display 2a and the P seat front display 2c may be the targets of the coordinated display. In other words, the center information display 2b does not have to be the target of the coordinated display.

[0112] <Summary of this embodiment> As described above, according to this embodiment, the control unit 11 of the display system ECU 5x is able to display camera images in coordination with the display areas of the multiple displays 2a to 2c that make up the PtoP display 2. In this case, the control unit 11 controls the display of the camera images by setting whether to display the camera images in coordination with at least two or more of the display areas of the displays 2a to 2c or to display the camera images on a single center information display 2b, depending on the autonomous driving level and the conditions inside or outside the vehicle.

[0113] When controlling the display of camera images on the displays 2a to 2c, the control unit 11 can change whether to display the camera images on some or all of the multiple displays 2a to 2c in cooperation with one another or on a single display, depending on the situation. This allows the camera images and other displays to be displayed appropriately to the occupants in various scenes where camera images are displayed.

[0114] In particular, when displaying in a coordinated manner on multiple displays 2a to 2c, the control unit 11 displays one camera image in a coordinated manner on the PtoP display 2, which is configured by continuously connecting the D seat front display 2a, which serves as the meter display area, the center information display 2b, and the P seat front display 2c, making it easier for occupants to check the various contents displayed in a coordinated manner.

[0115] Second Embodiment A second embodiment will be described with reference to Figures 21 and 22. The configuration of the in-vehicle display is not limited to the configuration of the P-to-P display 2 described in the first embodiment. For example, as shown in Figure 21, a meter display 2m in front of the D seat may be configured independently, and a display 2g in which a center information display 2b and a P seat front display 2c are integrated may be configured as a display device.

[0116] In the display configuration shown in Fig. 21, when the control unit 11 controls the display of a single display, the center information display 2b may be the display target, as shown by the hatching in the upper part of Fig. 22. The display 2c in front of the passenger seat may also be the display target. When the control unit 11 controls the display of multiple displays in a linked manner, the entire surface of the display 2g may be the display target in a linked manner, as shown by the hatching in the lower part of Fig. 22. This embodiment also achieves the same effects as the above-described embodiment.

[0117] Third Embodiment A third embodiment will be described with reference to Fig. 23 and Fig. 24. The configuration of the in-vehicle display may be, for example, a display 2h shown in Fig. 23. As shown in Fig. 23, the display 2h is a display device that is configured by integrating a center information display 2b with a D-seat front display 2a.

[0118] In the display configuration shown in Fig. 23, when the control unit 11 controls display of a single display, the display 2a in front of the D seat may be the display target, as shown by the hatching in the upper part of Fig. 24. The center information display 2b may also be the display target. When the control unit 11 controls linked display on multiple displays, the entire surface of the display 2h may be the linked display target, as shown by the hatching in the lower part of Fig. 24. This embodiment also achieves the same effects as the above-described embodiment.

[0119] (Fourth Embodiment) A fourth embodiment will be described with reference to Figures 25 and 26. As shown in Figure 25, the same application as in the previous embodiment can be applied to a case where the PtoP display 2 is composed of five displays 2a to 2e. As shown in Figure 25, the PtoP display 2 has a display 2a for displaying vehicle information and an electronic mirror display 2d in front of the D seat. As explained in the previous embodiment, the D seat front display 2a is a display device having a meter display area for displaying vehicle information SP. The electronic mirror display 2d is a display device that generally shows an image of the left rear of the vehicle.

[0120] The PtoP display 2 also includes a P-seat front display 2c and an electronic mirror display 2e in front of the P-seat. The P-seat front display 2c is a display device that normally displays entertainment information. The electronic mirror display 2e is a display device that displays an image of the right rear of the vehicle.

[0121] In the display configuration shown in Fig. 25, when the control unit 11 controls the display of a single display, the center information display 2b may be the display target, as shown by the hatching in the upper part of Fig. 26. Alternatively, any of the other displays 2a, 2b, 2d, and 2e may be the display target.

[0122] When electronic mirror displays 2d and 2e are provided, as shown by hatching in the lower part of Fig. 26, the control unit 11 may display camera images on the center information display 2b in cooperation with the electronic mirror displays 2d and 2e. The camera images may be displayed only on the electronic mirror displays 2d and 2e in cooperation with the center information display 2b. Note that, if the camera images are displayed in cooperation with the other displays 2a, 2b, and 2c, it is not necessary to display the camera images on the electronic mirror displays 2d and 2e in cooperation with the center information display 2b. Such an embodiment also achieves the same effects as the above-described embodiment.

[0123] Fifth Embodiment A fifth embodiment will be described with reference to FIG. 27 . In the above-described embodiments, a display system ECU 5x is configured using multiple ECUs 5. However, the display system ECU 5x may be configured as an HCU using a single ECU 5. An example of the hardware and software configuration in this case is shown in FIG. 27 . Each ECU 5 is equipped with an SoC 230, and a microcomputer is embedded in the SoC 230. The microcomputer embedded in the SoC 230 of the ECU 5 has a general-purpose OS 32 and a real-time OS 35 running on a hypervisor 231. A variety of applications 33 are configured to run on the general-purpose OS 32. The real-time OS 35 has higher real-time processing performance than the general-purpose OS 32. A meter application 36 is configured to run on the real-time OS 35. Even when such a configuration is adopted, the same configuration, functions, and effects as those of the above-described embodiments can be achieved.

[0124] (Other Embodiments) The present disclosure is not limited to the above-described embodiments, and for example, the following modifications or extensions are possible. In addition to the contents described in the claims, the present disclosure also includes the following contents. [1] A display control device (5x, 10) for a vehicle that enables camera video to be displayed by linking the display areas of multiple display devices (2, 2a to 2e, 2g, 2h, 2m), the display control device including a display control unit (11) that controls the display of the camera video by setting whether the display area of ​​the camera video is displayed on at least two or more of the multiple display devices in a linked manner or on a single display device, depending on an autonomous driving level and a situation inside or outside the vehicle.

[0125] [2] The vehicle display control device according to [1], further comprising a usage determination unit (11d) that determines whether the passenger seat display is being used, and the display control unit controls the display of the camera image by setting whether to display the image on the linked display or the single display depending on the determination result of the usage determination unit.

[0126] [3] A display control device for a vehicle according to [1] or [2], which includes an autonomous driving level determination unit (11c) that determines whether the autonomous driving level is autonomous driving or manual driving, and the display control unit controls the display of camera images by setting whether to display the images on the linked display or the single display depending on the determination result of the autonomous driving level determination unit.

[0127] [4] A display control device for a vehicle according to any one of [1] to [3], further comprising a periphery determination unit (11e) that determines whether or not a pedestrian, another vehicle, or a foreign object is present in the vicinity, wherein the display control unit controls the display of the camera image by setting whether to display the image on the linked display or on the single display depending on the determination result of the periphery determination unit.

[0128] [5] A display control device for a vehicle according to any one of [1] to [4], further comprising a tire display determination unit (11f) that determines whether or not to display tires, wherein the display control unit controls the display of the camera image by setting whether to display the linked display or the single display according to the determination result of the tire display determination unit.

[0129] [6] A display control system for a vehicle comprising: a plurality of display devices (2, 2a to 2e, 2g, 2h, 2m); and a display control unit (11) that controls the display of the camera images by setting whether the display area of ​​the camera images is displayed on the display areas of the plurality of display devices in a coordinated manner or on a single display device, depending on the autonomous driving level and the situation inside or outside the vehicle; and a display control device (5x, 10) that controls the display of the camera images by coordinating the display areas of the plurality of display devices.

[0130] [7] A display control method for a vehicle, wherein the display control unit (11) is configured to be able to control the display of the camera image by setting whether to display the display area of ​​the camera image on a plurality of display devices (2, 2a to 2e, 2g, 2h, 2m) in a coordinated manner or on a single display device, depending on the autonomous driving level and the situation inside or outside the vehicle, and the display control unit controls the display of the camera image by coordinating the plurality of display devices when the coordinated display setting is made.

[0131] [8] A display control program for a vehicle that, when configured to be able to control the display of the camera image by setting whether the display area of ​​the camera image is displayed on the multiple display devices in a coordinated manner or on a single display device depending on the autonomous driving level, the situation inside or outside the vehicle, causes the display control unit to execute a procedure for controlling the display of the camera image by coordinating the multiple display devices when the coordinated display setting is made.

[0132] The techniques described in this disclosure may be implemented by a special purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the techniques described in this disclosure may be implemented by a special purpose computer configured with a processor comprising one or more dedicated hardware logic circuits. Alternatively, the techniques described in this disclosure may be implemented by one or more special purpose computers configured with a processor comprising one or more hardware logic circuits in combination with a processor and memory programmed to perform one or more functions. Furthermore, a computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0133] Although the present disclosure has been described based on the above-described embodiment, it is understood that the present disclosure is not limited to the embodiment or the structure described in the embodiment. The present disclosure also encompasses various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more than one element, or less than one element, are also within the scope and spirit of the present disclosure.

Claims

1. A vehicle display control device (5x, 10) that enables camera images to be displayed by linking the display areas of multiple display devices (2, 2a to 2e, 2g, 2h, 2m), and that includes a display control unit (11) that controls the display of the camera images by setting the display areas of the camera images to be linked and displayed on at least two or more of the multiple display devices or on a single display device, depending on the autonomous driving level and the conditions inside or outside the vehicle.

2. A display control device for a vehicle as described in claim 1, which is provided with a usage determination unit (11d) that determines whether the passenger seat display device (2c, 2e) is being used, and the display control unit controls the display of the camera image by setting whether to display the image in the linked display or on the single display depending on the determination result of the usage determination unit.

3. A display control device for a vehicle as described in claim 1, which is provided with an autonomous driving level determination unit (11c) that determines whether the autonomous driving level is autonomous driving or manual driving, and the display control unit controls the display of the camera image by setting whether to display the image on the linked display or the single display depending on the determination result of the autonomous driving level determination unit.

4. A display control device for a vehicle as described in claim 1, which is provided with a surroundings determination unit (11e) that determines whether pedestrians, other vehicles, or foreign objects are present in the vicinity, and the display control unit controls the display of the camera image by setting whether to display the image on the linked display or on the single display depending on the determination result of the surroundings determination unit.

5. A display control device for a vehicle as described in claim 1, further comprising a tire display determination unit (11f) that determines whether or not to display tires, and the display control unit controls the display of the camera image by setting whether to display the image in the linked display or on the single display depending on the determination result of the tire display determination unit.

6. A vehicle display control system comprising: a plurality of display devices (2, 2a to 2e, 2g, 2h, 2m); a display control unit (11) that controls the display of the camera images by setting whether the display area of ​​the camera images is displayed on the display areas of the plurality of display devices in a coordinated manner or on a single display device depending on the autonomous driving level and the situation inside or outside the vehicle; and a display control device (5x, 10) that controls the display of the camera images by coordinating the display areas of the plurality of display devices.

7. A display control method for a vehicle, wherein the display control unit (11) is configured to be able to control the display of the camera image by setting whether to display the camera image in the display areas of multiple display devices (2, 2a to 2e, 2g, 2h, 2m) in a coordinated manner or on a single display device, depending on the autonomous driving level and the situation inside or outside the vehicle, and the display control unit controls the display of the camera image by coordinating the multiple display devices when the coordinated display setting is made.

8. A display control program for a vehicle that, when configured to be able to control the display of the camera image by a display control unit (11) by setting whether the display area of ​​the camera image is displayed on multiple display devices in a coordinated manner or on a single display device depending on the autonomous driving level and the situation inside or outside the vehicle, causes the display control unit to execute a procedure for controlling the display of the camera image by coordinating the multiple display devices when the coordinated display setting is made.

Images