Display control system

Abstract

The objective is to check the situation behind the vehicle while also getting a basic understanding of the conditions inside the vehicle. [Solution] The display control system (100) comprises a first imaging unit (20), a second imaging unit (30), a display unit (10), and a display control unit (41). The first imaging unit (20) photographs the rear of the moving object. The second imaging unit (30) photographs the inside of the moving object. The display unit (10) is located in a position visible to the driver of the moving object. The display control unit (41) displays at least one of the first image (21) generated using the first imaging unit (20) and the second image (31) generated using the second imaging unit (30) on the display unit (10). The display control unit (41) changes the display mode of at least one of the first image (21) and the second image (31) according to a predetermined state, and displays both the first image (21) and the second image (31) on the display unit (10) when the predetermined state satisfies a first condition.

Description

Technical Field

[0001] The present invention relates to a display control system.

Background Art

[0002] Patent Document 1 discloses an in-vehicle conversation support device capable of realizing smooth conversations between front-seat passengers and rear-seat passengers.

[0003] Patent Document 2 discloses an electronic mirror system that can fully utilize the functions of the device.

[0004] Patent Document 3 discloses an electronic mirror device applied to a rearview mirror, which can present significant information to passengers when the vehicle is stopped and can improve safety.

[0005] Patent Document 4 discloses a rearview electronic mirror and a control system that are easy to check the rear.

[0006] Patent Document 5 discloses an image processing device that enables only a partial area of a display screen to be used as a mirror.

[0007] Patent Document 6 discloses a vehicle display device that can display information necessary for driving at a position where the driver's line of sight is naturally directed during vehicle operation.

Prior Art Documents

Patent Documents

[0008]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

[0009] Generally, when using electronic mirrors that display images from cameras mounted at the rear of the vehicle, the electronic mirrors do not show the interior of the vehicle, making it difficult to understand what is happening inside.

[0010] For example, while Patent Document 2 describes a camera mounted near the side mirror, it is difficult to obtain a detailed view of the rear seats inside the vehicle using the electronic mirror. Furthermore, it is inconvenient for the driver to have to turn around to check on the rear seats.

[0011] For example, Patent Document 4 describes a system equipped with a camera to film the interior of the vehicle and another camera to film the rear of the vehicle, but it is cumbersome to constantly switch between cameras to monitor the interior and rear of the vehicle.

[0012] One example of a problem that this invention aims to solve is the ability to easily understand the situation inside a vehicle while simultaneously checking the situation behind the vehicle. [Means for solving the problem]

[0013] The invention described in claim 1 is, The first camera unit photographs the rear of the moving object, A second imaging unit for photographing the inside of the aforementioned moving body, The driver of the aforementioned mobile body has a display unit located in a position where he can see it, The system includes a display control unit that displays at least one of a first image generated using the first imaging unit and a second image generated using the second imaging unit on the display unit, The display control unit changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition. This is a display control system.

[0014] One aspect of the present invention is a display unit located at a position visible to the driver of the moving body, a display control unit that displays at least one of a first image generated using a first imaging unit and a second image generated using a second imaging unit on the display unit, The display control unit changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition. This is an electronic mirror.

[0015] One aspect of the present invention is one or more computers that implement a display control system image the rear of the moving body, image the inside of the moving body, display at least one of a first image generated by imaging the rear of the moving body and a second image generated by imaging the inside of the moving body on a display unit located at a position visible to the driver of the moving body, change the display mode of at least one of the first image and the second image according to a predetermined state, and display both the first image and the second image on the display unit when the predetermined state satisfies a first condition. This is a control method for a display control system.

[0016] One aspect of the present invention is to one or more computers that implement a display control system a procedure for imaging the rear of the moving body, a procedure for imaging the inside of the moving body, A procedure of displaying at least one of a first image generated by photographing the rear of the moving body and a second image generated by photographing the inside of the moving body on a display unit located at a position visible to the driver of the moving body. A program for executing a procedure of changing a display mode of at least one of the first image and the second image according to a predetermined state, and displaying both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

Brief Description of the Drawings

[0017] [Figure 1] It is a block diagram showing the functions of the display control system according to the first embodiment. [Figure 2] It is a diagram showing a vehicle realizing the display control system according to the first embodiment. [Figure 3] It is a diagram showing a state where both the first image and the second image are displayed on the display unit. [Figure 4] It is a diagram showing an example of the hardware configuration of the server. [Figure 5] It is a flowchart when the display control unit according to the first embodiment displays both the first image and the second image on the display unit. [Figure 6] It is a flowchart when the display control unit according to the first embodiment changes the display mode of the second image. [Figure 7] It is a block diagram showing the functions of the display control system according to the second embodiment. [Figure 8] It is a diagram showing a state where the second image is displayed small on the display unit. [Figure 9] It is a diagram showing the second image in which the passengers are emphasized and displayed. [Figure 10] It is a block diagram showing the functions of the display control system according to the fourth embodiment. [Figure 11] It is a diagram showing a state where the passengers included in the second image are displayed on the display unit in an icon mode. [Figure 12] It is a diagram showing another mode of the icon. [Figure 13] This is a block diagram illustrating the function of the electronic mirror 1 according to the sixth embodiment. [Modes for carrying out the invention]

[0018] Embodiments of the present invention will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted as appropriate.

[0019] In the following description, each component of each device refers to a functional block, not a hardware-level configuration. Each component of each device is realized through any combination of hardware and software, centered around the CPU, memory, programs loaded into memory, storage media such as a hard disk that stores those programs, and a network connection interface of any computer. Furthermore, there are various variations in the implementation method and the device itself.

[0020] (First Embodiment) Figure 1 is a block diagram showing the functions of the display control system 100 according to the first embodiment. Figure 2 is a diagram showing a vehicle 5 that implements the display control system 100 according to the first embodiment. The display control system 100 according to the first embodiment will be described using Figures 1 and 2.

[0021] The display control system 100 comprises a display unit 10, a first imaging unit 20, a second imaging unit 30, and a server 40. In the first embodiment, the server 40 is located outside the mobile body. The server 40 may be a so-called cloud server. In the first embodiment, the mobile body is a vehicle 5.

[0022] (First Photography Section 20) As shown in Figure 2, in the first embodiment, the first imaging unit 20 is located outside the vehicle. The first imaging unit 20 is located at the rear of the vehicle 5. Alternatively, the first imaging unit 20 may be located inside the vehicle and facing outwards. The interior of the vehicle refers to the area where the driver and passengers are located. The exterior of the vehicle refers to the area outside the vehicle body where the driver and passengers are not located.

[0023] The first imaging unit 20 photographs the rear of the vehicle 5. The first imaging unit 20 photographs a predetermined range θ1 centered on the first imaging unit 20. The first imaging unit 20 may be movable and may be able to swing around the height axis of the vehicle 5.

[0024] The first imaging unit 20 generates a first image. The first image is an image of the rear of the vehicle, captured by the first imaging unit 20. The first image is generated by capturing the rear of the vehicle 5. The first image is an image generated using the first imaging unit 20 and includes at least one of a still image and a video.

[0025] As shown in Figure 1, the first imaging unit 20 is configured to communicate with the server 40 via the communication network 101. The communication network 101 includes, for example, communication over a 4G or 5G line. The first imaging unit 20 is configured to include an external communication unit (not shown) for configuring the communication network 101. The first imaging unit 20 transmits information related to the first image to the server 40.

[0026] (Second Photography Section 30) As shown in Figure 2, the second imaging unit 30 is installed inside the vehicle. The second imaging unit 30 may be brought in from outside the vehicle 5, or it may be pre-installed in the vehicle 5. The second imaging unit 30 is installed, for example, on the ceiling of the vehicle 5. The second imaging unit 30 is installed, for example, on the ceiling between the driver's seat 5a and the passenger seat 5b.

[0027] The second camera unit 30 photographs the interior of the vehicle. The second camera unit 30 photographs a predetermined range θ2 centered on the second camera unit 30. For example, the second camera unit 30 photographs the interior of the vehicle so that the rear seats 5c of the vehicle 5 are photographed in their entirety. For example, the second camera unit 30 photographs passengers sitting in the rear seats 5c. The second camera unit 30 may be movable and may be able to swing around the height axis of the vehicle 5.

[0028] The second imaging unit 30 generates a second image. The second image is an image of the interior of the vehicle, taken by the second imaging unit 30. The second image is an image generated by taking images of the interior of the vehicle 5. The second image is an image generated using the second imaging unit 30 and includes at least one of a still image and a video.

[0029] As shown in Figure 1, the second imaging unit 30 is configured to communicate with the server 40 via the communication network 101. The second imaging unit 30 includes an external communication unit (not shown) for configuring the communication network 101. The second imaging unit 30 transmits information related to the second image to the server 40.

[0030] (Display section 10) In the first embodiment, the display unit 10 is a so-called electronic mirror. The display unit 10 displays at least one of a first image and a second image. As shown in Figure 2, the display unit 10 is located in a position visible to the driver of the vehicle 5. The display unit 10 may be installed, for example, on the ceiling between the driver's seat 5a and the passenger seat 5b. The display unit 10 may be installed, for example, in a position closer to the driver's seat 5a than to the passenger seat 5b. The display unit 10 may be installed, for example, in a position closer to the driver's seat 5a than to the rear seat 5c.

[0031] As shown in Figure 1, the display unit 10 includes a speaker as an audio output unit 11 and a microphone as an audio input unit 12. A transducer may also be provided inside the display unit 10 (=electronic mirror), and this transducer may be used to realize the speaker and microphone of the display unit 10.

[0032] Furthermore, a microphone (not shown) may be provided near the rear seat 5c, and the sound input to the microphone may be output from the sound output unit 11. Also, a speaker (not shown) may be provided near the rear seat 5c, and the driver's voice input to the sound input unit 12 may be output from the speaker. In this way, the system may be configured so that the driver and the passenger in the rear seat 5c can communicate via the display unit 10 (=electronic mirror).

[0033] As shown in Figure 1, the display unit 10 is configured to communicate with the server 40 via the communication network 101. The display unit 10 includes an external communication unit (not shown) for configuring the communication network 101.

[0034] (Display control unit 41) The server 40 includes a display control unit 41. The display control unit 41 displays at least one of the first image generated using the first imaging unit 20 and the second image generated using the second imaging unit 30 on the display unit 10. The display control unit 41 may display only the first image on the display unit 10, or only the second image on the display unit 10.

[0035] The display control unit 41 changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit 10 when the predetermined state satisfies the first condition.

[0036] (Predetermined state) The predetermined state includes at least one of the following: the driving state of the vehicle 5, the state of the rear of the vehicle 5, the state of the passengers inside the vehicle 5, and the state of the driver.

[0037] (Vehicle 5's driving status) The driving state of vehicle 5 includes at least one of the vehicle's speed state and the condition of the road on which vehicle 5 is traveling. The vehicle's speed state includes either the state relating to how fast vehicle 5 is traveling or the state relating to whether the vehicle's speed state is stable or not. The condition of the road on which vehicle 5 is traveling includes at least one of the following: for example, whether vehicle 5 is traveling on a public road, whether vehicle 5 is traveling on a highway, or what the speed limit of the road is.

[0038] Furthermore, the driving state of vehicle 5 includes, for example, whether or not vehicle 5 is driving erratically. Also, the driving state of vehicle 5 (= the speed of vehicle 5) includes, for example, whether the vehicle is stopped or whether vehicle 5 is caught in traffic.

[0039] (Condition of the rear of vehicle 5) The condition behind vehicle 5 includes, for example, whether or not another vehicle is present behind vehicle 5. The condition behind vehicle 5 also includes, for example, whether or not the distance between vehicle 5 and the vehicle behind it is within a predetermined distance. Whether or not another vehicle is present behind vehicle 5 and whether or not the distance to the vehicle behind it is within a predetermined distance may be determined using at least one of the first image and the vehicle's traffic information (location information).

[0040] (Condition of passengers inside vehicle 5) The state of the occupants inside vehicle 5 includes, for example, at least one of the following: whether or not the occupants are speaking, their posture (sleeping, awake, etc.), their facial expression (laughing, sad, etc.), and their needs (wanting to go to the toilet, hungry, etc.). For example, the state of the occupants inside vehicle 5 can be identified by analyzing the second image.

[0041] (Driver status) The driver's state includes, for example, at least one of the following: the degree to which the driver pays attention to the electronic mirror (display unit 10), the driver's fatigue, and the driver's physical condition.

[0042] Figure 3 shows the display unit 10 with both the first image 21 and the second image 31 displayed. The display control unit 41 displays both the first image 21 and the second image 31 on the display unit 10 when a predetermined state satisfies the first condition.

[0043] When the predetermined state satisfies the first condition, it includes at least one of the following: when the engine of vehicle 5 is started, when a passenger speaks, and when a passenger has not spoken for a predetermined period of time (e.g., 5 minutes). Whether or not a passenger is speaking may be determined by image analysis using the second image, or by using the passenger's voice information input to the microphone.

[0044] (Example hardware configuration) Figure 4 shows an example of the hardware configuration of server 40. Server 40 has a bus 1010, a processor 1020, memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0045] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0046] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0047] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0048] The storage device 1040 is an auxiliary storage device implemented as a removable media such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or memory card, or as ROM (Read Only Memory), and has a recording medium. The recording medium of the storage device 1040 stores program modules that implement each function of the server 40 (for example, the display control unit 41). The processor 1020 reads these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. Alternatively, the storage device 1040 may function as a storage unit (not shown) connected to the server 40.

[0049] The input / output interface 1050 is an interface for connecting the server 40 with various input / output devices.

[0050] The network interface 1060 is an interface for connecting the server 40 to a network. This network may be, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method by which the network interface 1060 connects to the network may be wireless or wired. The server 40 may communicate with the display unit 10, the first imaging unit 20, and the second imaging unit 30 via the network interface 1060.

[0051] Furthermore, the parts of the display unit 10, the first imaging unit 20, and the second imaging unit 30 that are related to communication or control may also be implemented with the same hardware configuration as the server 40.

[0052] (Example of operation of the first embodiment 1) Figure 5 is a flowchart showing how the display control unit 41 according to the first embodiment displays both the first image 21 and the second image 31 on the display unit 10.

[0053] In step S10, the display control unit 41 (=server 40) acquires information about the first image from the first imaging unit 20.

[0054] In step S20, the display control unit 41 (=server 40) acquires information about the second image from the second imaging unit 30.

[0055] In step S30, the display control unit 41 determines whether or not the passenger in the rear seat 5c has spoken for a predetermined period of time. The display control unit 41 may determine whether or not the passenger in the rear seat 5c has spoken using voice information input to a microphone (not shown) mounted near the rear seat 5c. Alternatively, the display control unit 41 may determine whether or not the passenger in the rear seat 5c has spoken using a second image.

[0056] If the display control unit 41 determines that the passenger in the rear seat 5c has not spoken for a predetermined period of time (YES in step S30), the process proceeds to step S40. If the display control unit 41 determines that the passenger in the rear seat has spoken for a predetermined period of time (NO in step S30), the process returns to step S30 and the processing in step S30 is repeated.

[0057] Then, in step S40, the display control unit 41 displays both the first image 21 and the second image 31 on the display unit 10. After that, the process ends.

[0058] In step S30, instead of the display control unit 41 determining whether the passenger in the rear seat 5c has not spoken for a predetermined period of time, the flow may also be to determine whether the engine of the vehicle 5 has started or whether the passenger has spoken.

[0059] (Example of operation of the first embodiment 2) Figure 6 is a flowchart showing how the display control unit 41 according to the first embodiment changes the display mode of the second image 31. Operation Example 2 is the same as the flowchart of Operation Example 1, but with the addition of steps S50 and S60.

[0060] In step S50, following step S40, the display control unit 41 determines whether the vehicle 5's speed is stable. Whether the vehicle's speed is stable means that the driver has been driving within a predetermined speed range for a certain period of time. For example, if the driver has been driving within the range of 55 km / h to 65 km / h for 10 minutes continuously, the vehicle's speed is considered stable. Alternatively, whether the vehicle's speed is stable may be determined by whether the driver has been driving within a predetermined acceleration range for a certain period of time.

[0061] If the display control unit 41 determines that the vehicle's speed is stable (YES in step S50), the process proceeds to step S60. If the display control unit 41 determines that the vehicle's speed is not stable (NO in step S50), the process ends. If NO is obtained in step S50, the process may return to step S50 and repeat the process in step S50.

[0062] In step S60, the display control unit 41 changes the display mode of the second image 31 so that the size of the second image 31 is larger than the size of the second image 31 displayed in step S40. Note that steps S50 and S60 do not necessarily have to be performed after step S40, and may be performed after, for example, steps S10 and S20.

[0063] As described above, according to the first embodiment, the display control system 100 comprises a first imaging unit 20, a second imaging unit 30, a display unit 10, and a display control unit 41. The display control unit 41 displays both the first image 21 and the second image 31 on the display unit 10 when a predetermined state satisfies the first condition.

[0064] When a predetermined state satisfies the first condition, both the first image 21 and the second image 31 are displayed on the display unit 10, allowing the driver to grasp the situation behind the vehicle 5 and the situation of the passengers inside the vehicle 5 at a glance on the display unit 10. Furthermore, the driver can grasp the situation of the passengers inside the vehicle 5 (rear seats) without having to turn around or call out to the passengers in the back seat, thus reducing inconvenience. Therefore, the driver can easily grasp the situation inside the vehicle 5 while checking (or simultaneously checking) the situation behind the vehicle 5.

[0065] Furthermore, the display control unit 41 changes the display mode of at least one of the first image 21 and the second image 31 according to a predetermined state. This makes it possible to effectively display the first image 21 and the second image 31 on the display unit 10 according to various states (predetermined states).

[0066] Furthermore, the predetermined state when the first condition is met includes at least one of the following: when the engine of the vehicle 5 is started, when a passenger speaks, and when a passenger does not speak for a predetermined period of time. For example, if a driver does not speak for a predetermined period of time, they may worry whether the passenger is feeling unwell. When a passenger does not speak for a predetermined period of time (when the predetermined state meets the first condition), displaying both the first image 21 and the second image 31 on the display unit 10 allows the driver to visually check the passenger's condition and easily grasp their state. As a result, the driver can concentrate on driving with peace of mind.

[0067] Furthermore, the predetermined state includes at least one of the following: the driving state of the vehicle 5, the state of the rear of the vehicle 5, the state of the occupants inside the vehicle 5, and the state of the driver. This allows the first image 21 and the second image 31 to be effectively displayed on the display unit 10 according to various situations.

[0068] Furthermore, the driving status of vehicle 5 includes at least one of the vehicle's speed and the condition of the road on which vehicle 5 is traveling. This allows the first image 21 and the second image 31 to be effectively displayed on the display unit 10 according to various conditions related to the vehicle's speed and the road on which vehicle 5 is traveling.

[0069] The display unit 10 also includes a speaker and a microphone. This allows the driver to communicate with rear-seat passengers via the display unit 10. Furthermore, by mounting the speaker and microphone on the display unit 10 (electronic mirror) in the driver's line of sight, the directivity of the driver's voice can be improved. As a result, the driver and rear-seat passengers can communicate effectively. In addition, by using a separate speaker in addition to the existing speaker, the driver and passengers can communicate without interfering with the existing speaker system.

[0070] (Second Embodiment) Figure 7 is a block diagram showing the functions of the display control system 100 according to the second embodiment. Unlike the first embodiment, the server 40 according to the second embodiment further includes a vehicle distance acquisition unit 42.

[0071] The inter-vehicle distance acquisition unit 42 acquires inter-vehicle distance information relating to the distance (inter-vehicle distance) between vehicle 5 and the vehicle behind vehicle 5. The inter-vehicle distance acquisition unit 42 may acquire inter-vehicle distance information through the first imaging unit 20, or it may acquire inter-vehicle distance information using vehicle position information and traffic information. In the predetermined state according to the second embodiment, the state behind vehicle 5 includes a state in which the inter-vehicle distance between vehicle 5 and the vehicle behind vehicle 5 is less than or equal to a predetermined distance.

[0072] Figure 8 shows the second image 31 displayed in a smaller size on the display unit 10. When the distance between vehicles falls below a predetermined distance, the display control unit 41 changes the display mode of the second image 31 so that the size of the second image 31 displayed on the display unit 10 becomes smaller or the second image 31 is not displayed at all. The second image 31 in Figure 8 is displayed smaller on the display unit 10 compared to the second image 31 in Figure 3.

[0073] When the distance between vehicle 5 and the vehicle behind it is small (including when being tailgated), the driver must concentrate on driving. By displaying the second image 31 in a smaller size or not displaying the second image 31 at all, the driver's concentration on the second image 31 can be reduced. As a result, the driver's concentration on driving is improved.

[0074] Furthermore, in another embodiment, the inter-vehicle distance acquisition unit 42 may acquire rear vehicle information. Rear vehicle information is information regarding the presence or absence of a vehicle behind vehicle 5. The inter-vehicle distance acquisition unit 42 may acquire rear vehicle information through the first imaging unit 20, or it may acquire rear vehicle information using vehicle position information and traffic information. In the predetermined state according to the second embodiment, the state behind vehicle 5 includes a state in which there is no vehicle behind vehicle 5.

[0075] When there is no vehicle behind vehicle 5, the display control unit 41 changes the display mode of the second image 31 displayed on the display unit 10 so that the size of the second image 31 is increased. When there is no vehicle behind vehicle 5, it may not be a problem to increase the level of focus on the display unit 10 (=second image 31). By displaying the second image 31 larger when there is no vehicle behind vehicle 5, the situation of the passengers in the rear seats can be effectively grasped.

[0076] In the second embodiment, the display control unit 41 may output sound effects depending on a predetermined state. The display control unit 41 may output a warning sound or output a warning mark to the display unit 10. For example, if the distance between vehicle 5 and the vehicle behind it falls below a predetermined distance, the display control unit 41 may output a warning sound from the speaker. This makes it easier for the driver to notice that the distance to the vehicle behind is getting close.

[0077] (Third embodiment) The third embodiment differs in function from the display control unit 41 of the first embodiment. The basic configuration is the same as in Figure 1. In a predetermined state according to the third embodiment, the state of the occupant in the vehicle 5 includes the state in which the occupant is speaking. When the occupant is speaking, the display control unit 41 according to the third embodiment changes the display mode of the second image 31 so that the occupant 6 included in the second image 31 is highlighted.

[0078] Figure 9 shows the second image 31 in which passenger 6a is highlighted. As shown in Figure 9, the display control unit 41 may display the outer frame 7 of passenger 6a who is speaking on the display unit 10 with a thicker outline. The display control unit 41 may also display a highlighting icon 8 above the speaking passenger 6a. Although not shown, the display control unit 41 may also highlight passenger 6a by coloring its outer frame (for example, red).

[0079] In this way, the display control unit 41 highlights the passenger 6a who is speaking and displays it in the second image 31, allowing the driver to quickly recognize which of the multiple passengers 6 is speaking.

[0080] (Fourth Embodiment) Figure 10 is a block diagram showing the functions of the display control system 100 according to the fourth embodiment. Unlike the first embodiment, the server 40 according to the fourth embodiment further includes a driver information acquisition unit 43. The driver information acquisition unit 43 acquires driver status information. The driver status information includes information on whether or not the driver's attention level is reduced.

[0081] The driver information acquisition unit 43 may acquire driver status information by performing image analysis (for example, by analyzing the driver's gaze) using a camera (or second imaging unit 30) mounted inside the vehicle 5. The driver information acquisition unit 43 may also acquire driver status information using, for example, the driver's vital information (blood pressure, pulse rate, amount of sweat, etc.) obtained from in-vehicle equipment (steering wheel, seat belt, etc.) or devices worn by the driver.

[0082] In the predetermined state according to the fourth embodiment, the driver's state includes a state in which the driver's attention is reduced. When the driver's attention is reduced, the display control unit 41 according to the fourth embodiment changes the display mode of the second image so that the second image is not displayed. This suppresses the degree of concentration on the second image and allows the driver to concentrate on the first image (the image of the rear of the vehicle) when the driver's attention is reduced. This helps to reduce traffic accidents.

[0083] (Fifth embodiment) Figure 11 shows how the passenger 6 included in the second image 31 is displayed on the display unit 10 in icon form. The display control unit 41 according to the fifth embodiment displays the passenger 6 included in the second image 31 on the display unit 10 in icon form. The icons 44 represent, for example, the emotions of the passenger 6, such as joy, anger, sadness, etc. The display control unit 41 switches the actual image of the passenger 6 (second image 31) to an icon 44, for example, while driving. The display control unit 41 switches the image of the passenger 6 (see area A in Figure 11) to at least one of the icons 44. The display control unit 41 may switch only the face portion of the passenger 6 in the image of the passenger 6 to an icon 44, or it may switch the entire passenger 6 to an icon 44.

[0084] The display control unit 41 changes the icon appearance according to the state of the passenger 6. The passenger's state includes, for example, a state in which the passenger is happy, angry, sad, and enjoying themselves. For example, when passenger 6 is crying, the display control unit 41 may change the icon to one 44a that represents sadness.

[0085] Figure 12 shows another embodiment of icon 44. Icon 44 may represent not only the passenger's emotions (joy, anger, sadness, etc.) but also the passenger's requests. The passenger's state includes, in addition to the above states of joy, anger, sadness, etc., the passenger wanting to go to the toilet, the passenger being hungry, the passenger feeling hot, and the passenger feeling cold.

[0086] For example, icon 44w indicates that passenger 6 wants to go to the toilet, and icon 44x indicates that passenger 6 is hungry. In addition to icons representing facial expressions, icons that suggest passenger 6's needs (for example, toilet icon 45a and food icon 45b) may also be displayed at the same time.

[0087] The predetermined state according to the fifth embodiment includes the driving state of the vehicle 5. The driving state of the vehicle 5 includes the state in which the vehicle 5 is stopped. When the vehicle 5 is stopped, the display control unit 41 changes the display mode of the second image 31 so that the passenger 6, which is displayed in icon form, is displayed as a real image. The display control unit 41 may determine whether the vehicle 5 is stopped by whether the handbrake is functioning or by detecting the speed of the vehicle.

[0088] As described above, according to the fifth embodiment, the display control unit 41 displays the passenger 6 included in the second image 31 in icon form on the display unit 10. This makes it possible to suppress the driver's concentration on the display unit 10. Therefore, it is possible to suppress the occurrence of traffic accidents.

[0089] Furthermore, the display control unit 41 changes the icon configuration according to the status of the passenger 6. This allows the driver to easily understand the status of the passenger 6.

[0090] Furthermore, when the vehicle 5 is stopped, the display control unit 41 changes the display mode of the second image 31 so that the passenger 6, which is displayed as an icon, is displayed as a real image. When the vehicle 5 is stopped, there is no problem even if the driver concentrates on looking at the display unit 10 (second image 31). With the above configuration, the driver can safely understand the status of the passenger 6.

[0091] (Sixth Embodiment) Figure 13 is a block diagram illustrating the functions of the electronic mirror 1 according to the sixth embodiment. The electronic mirror 1 according to the sixth embodiment comprises a display unit 10 and a display control unit 41. Similar to the first embodiment, the display unit 10 is positioned in a location visible to the driver of the vehicle 5. In the sixth embodiment, the display unit 10 includes the display portion of the electronic mirror 1. Similar to the first embodiment, the display control unit 41 displays at least one of the first image generated using the first imaging unit 20 and the second image generated using the second imaging unit 30 on the display unit 10. Similar to the first embodiment, the display control unit 41 changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit 10 when the predetermined state satisfies the first condition.

[0092] The display control unit 41 may be mounted inside the display unit 10 or outside the display unit 10. If the display control unit 41 is mounted outside the display unit 10, it is located inside the vehicle and does not communicate with a server or other device located outside the vehicle. The display control unit 41 communicates with the display unit 10, the first imaging unit 20, and the second imaging unit 30, etc., which are mounted in the vehicle.

[0093] As described above, according to the sixth embodiment, the electronic mirror 1 can be realized in a configuration that does not use a server 40.

[0094] In the sixth embodiment, the first imaging unit 20 photographs the rear of the vehicle 5, and the second imaging unit 30 photographs the interior of the vehicle 5. This makes it possible to realize an electronic mirror 1 that allows the driver to check the situation behind the vehicle while also easily understanding what is happening inside the vehicle.

[0095] (modified version) As a variation, the display control unit 41 may detect that the parking brake is functioning and display only the second image on the display unit 10.

[0096] Furthermore, when a vehicle is being tailgated (when the distance between vehicles repeatedly decreases and increases), the display control unit 41 may display only the first image on the display unit 10.

[0097] For example, if there are multiple passengers 6 in the rear seats, the display control unit 41 may display them one by one rather than displaying all of the passengers 6 simultaneously on the display unit 10 (second image 31). In this case, the display control unit 41 may switch to display each passenger 6 according to who is speaking.

[0098] The embodiments described above with reference to the drawings are examples of the present invention, and various other configurations can also be adopted.

[0099] The display control unit 41 according to the first embodiment may be included in the display unit 10. In this case, the display control system 100 can be realized even without a server 40.

[0100] Furthermore, the display control system 100 is implemented using one or more computers, including the display unit 10, the first imaging unit 20, the second imaging unit 30, and the server 40. The number of computers used to implement the display control system 100 is arbitrary.

[0101] The display unit 10 according to the first embodiment may be a head-up display or other display that can be positioned in a location visible to the driver.

[0102] Examples of reference formats are provided below. 1. The first imaging unit, which photographs the rear of the moving object, A second imaging unit for photographing the inside of the aforementioned moving body, The driver of the aforementioned mobile body has a display unit located in a position where he can see it, The system includes a display control unit that displays at least one of a first image generated using the first imaging unit and a second image generated using the second imaging unit on the display unit, A display control system comprising: a display control unit that changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition. 2. In the display control system described in 1., A display control system that, when the predetermined state satisfies the first condition, includes at least one of the following: when the engine of the mobile body is started, when the passenger speaks, and when the passenger does not speak for a predetermined period of time. 3. In the display control system described in 1. or 2., A display control system in which the predetermined state includes at least one of the following: the driving state of the mobile body, the state of the rear of the mobile body, the state of the occupants inside the mobile body, and the state of the driver. 4. In the display control system described in 3., A display control system in which the driving state of the moving body includes at least one of the speed of the moving body and the condition of the road on which the moving body is traveling. 5. In the display control system described in 3. or 4, The state of the rear of the moving body includes a state in which the distance between the moving body and the vehicle behind it is less than or equal to a predetermined distance. A display control system in which, when the distance between vehicles falls below a predetermined level, the display control unit changes the display mode of the second image so that the size of the second image displayed on the display unit becomes smaller or the second image is not displayed. 6. In the display control system described in any one of 3 to 5, The state behind the moving body includes the state in which there is no vehicle behind the moving body. A display control system in which, when there is no vehicle behind the moving body, the display control unit changes the display mode of the second image so that the size of the second image displayed on the display unit increases. 7. In the display control system described in any one of 3 to 6, The state of the passenger inside the mobile vehicle includes the state in which the passenger is speaking. A display control system in which, when the passenger is speaking, the display control unit changes the display mode of the second image so that the passenger included in the second image is highlighted. 8. In the display control system described in any one of 3 to 7, The state of the driver includes a state in which the driver's attention is reduced. A display control system in which, when the driver's attention is reduced, the display control unit changes the display mode of the second image so as not to display the second image. 9. In the display control system described in any one of items 1 to 8, The display control unit is a display control system that displays the passengers included in the second image in icon form on the display unit. In the display control system described in 10.9, The display control unit is a display control system that changes the icon configuration according to the status of the passenger. 11. In the display control system described in 9. or 10., The predetermined state includes the state of movement of the moving body, The state of the moving body includes the state in which the moving body is stationary. A display control system in which, when the moving body is stopped, the display control unit changes the display mode of the second image so that the passenger, who is displayed in icon form, is displayed as a real image. 12. In a display control system described in any one of items 1 to 11, The display control unit is a display control system that outputs sound effects according to the predetermined state. 13. In a display control system described in any one of items 1 to 12, The display unit is a display control system including a speaker and a microphone. 14. A display unit located in a position visible to the driver of the mobile device, The system includes a display control unit that displays at least one of a first image generated using a first imaging unit and a second image generated using a second imaging unit on the display unit, The display control unit changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition, an electronic mirror. 15. In the electronic mirror of 14, The first imaging unit photographs the rear of the moving object, The second imaging unit is an electronic mirror that images the inside of the moving body. 16. One or more computers that implement a display control system, Take a picture of the rear of the moving object. The inside of the aforementioned moving body is photographed, At least one of the first image generated by photographing the rear of the moving body and the second image generated by photographing the inside of the moving body is displayed on a display unit located in a position visible to the driver of the moving body. A control method for a display control system, which changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition. 17. One or more computers that implement the display control system, Procedure for photographing the rear of a moving object, The procedure for photographing the inside of the aforementioned moving object, A procedure for displaying at least one of a first image generated by photographing the rear of the moving body and a second image generated by photographing the inside of the moving body on a display unit located in a position visible to the driver of the moving body. A program for causing a program to perform a procedure for changing the display mode of at least one of the first image and the second image according to a predetermined state, and for displaying both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

[0103] This application claims priority based on Japanese Patent Application No. 2023-018901, filed on 10 February 2023, and incorporates all of its disclosures herein. [Explanation of symbols]

[0104] 1. Electronic mirror 5 vehicles 5a Driver's seat 5b passenger seat 5c Rear Seat 6 passengers 10 Display 20 1st Photography Department 21 Image 1 30. Second Photography Department 31 Image 2 40 servers 41 Display Control Unit 42 Inter-vehicle distance acquisition section 43 Driver Information Acquisition Unit 44 icons 100 Display Control System

Claims

[Claim 1] The first camera unit photographs the rear of the moving object, A second imaging unit for photographing the inside of the aforementioned moving body, The driver of the aforementioned mobile body has a display unit located in a position where he can see it, The system includes a display control unit that displays at least one of a first image generated using the first imaging unit and a second image generated using the second imaging unit on the display unit, A display control system comprising: a display control unit that changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

Images