Display control system, electronic mirror, control method for display control system, and program

Abstract

A display control system (100) comprises a first imaging unit (20), a second imaging unit (30), a display unit (10), and a display control unit (41). The first imaging unit (20) images toward the rear of a moving body. The second imaging unit (30) images inside the moving body. The display unit (10) is in a position visible to a driver of the moving body. The display control unit (41) displays at least one of a first image (21) generated using the first imaging unit (20) and a second image (31) generated using the second imaging unit (30) on the display unit (10). The display control unit (41) changes a display mode of at least one of the first image (21) and the second image (31) in accordance with a prescribed state, and if the prescribed state satisfies a first condition, displays both the first image (21) and the second image (31) on the display unit (10).

Description

[Technical Field]

[0001] The present invention relates to a display control system, an electronic mirror, a control method for a display control system, and a program. [Background technology]

[0002] Patent Document 1 discloses an in-vehicle conversation support device that can realize smooth conversation between front seat passengers and rear seat passengers.

[0003] Patent Document 2 discloses an electronic mirror system that can fully utilize the functions of the device.

[0004] Patent Document 3 discloses an electronic mirror device that is applied to a rearview mirror and that can present significant information to occupants when the vehicle is stopped, thereby improving safety.

[0005] Patent Document 4 discloses an electronic rearview mirror and control system that makes it easy to check behind the driver.

[0006] Patent Document 5 discloses an image processing device that allows only a partial area of ​​a display screen to be used as a mirror.

[0007] Patent Document 6 discloses a vehicle display device that can display information necessary for driving at a position where the driver's line of sight is naturally directed when driving a vehicle. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Patent Publication No. 2021-9234 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-165292 [Patent Document 3] Japanese Patent Application Publication No. 2018-69862 [Patent Document 4] Japanese Patent Application Laid-Open No. 2017-47868 [Patent Document 5] Japanese Patent Application Publication No. 2017-34453 [Patent Document 6] Japanese Patent Application Publication No. 1-289725 Summary of the Invention [Problem to be solved by the invention]

[0009] Generally, when using an electronic mirror that displays images from a camera mounted at the rear of a vehicle, the electronic mirror does not show the interior of the vehicle, making it difficult to grasp the situation inside the vehicle.

[0010] For example, in Patent Document 2, a camera is mounted near the side mirror, but it is difficult to grasp the situation in the rear seats of the vehicle in detail using the electronic mirror. Also, it is troublesome for the driver to take the trouble to turn around and check the situation in the rear seats.

[0011] For example, in Patent Document 4, a camera that takes pictures of the inside of the vehicle and a camera that takes pictures of the rear of the vehicle are installed, but it is troublesome to switch between the cameras one by one to grasp the situation inside and behind the vehicle.

[0012] One example of a problem that the present invention aims to solve is to easily grasp the situation inside the vehicle while checking the situation behind the vehicle. [Means for solving the problem]

[0013] The invention described in claim 1 is a first imaging unit that images the rear of the moving object; a second imaging unit that images the inside of the moving body; a display unit located at a position visible to a driver of the vehicle; a display control unit that displays at least one of a first image generated using the first imaging unit and a second image generated using the second imaging unit on the display unit, The display control unit is a display control system that changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

[0014] The invention described in claim 14 is a display unit located at a position visible to a driver of the vehicle; a display control unit that displays at least one of a first image generated using a first imaging unit and a second image generated using a second imaging unit on the display unit, The display control unit is an electronic mirror that changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

[0015] The invention described in claim 16 is One or more computers implementing a display control system, Photograph the rear of the moving object, Taking an image of the inside of the moving body, displaying at least one of a first image generated by photographing the rear of the moving body and a second image generated by photographing the inside of the moving body on a display unit located at a position visible to a driver of the moving body; A control method for a display control system that changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

[0016] The invention described in claim 17 is One or more computers that implement the display control system Procedure for photographing the rear of a moving object, a step of photographing the inside of the moving body; a step of displaying at least one of a first image generated by photographing the rear of the moving body and a second image generated by photographing the inside of the moving body on a display unit located at a position visible to a driver of the moving body; The program executes a procedure for changing the display mode of at least one of the first image and the second image in accordance with a predetermined state, and displaying both the first image and the second image on the display unit when the predetermined state satisfies a first condition. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a block diagram showing functions of a display control system according to a first embodiment. [Figure 2] 1 is a diagram showing a vehicle in which a display control system according to a first embodiment is realized. [Figure 3] FIG. 2 is a diagram showing how both a first image and a second image are displayed on a display unit. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server. [Figure 5] 4 is a flowchart when the display control unit according to the first embodiment causes the display unit to display both the first image and the second image. FIG. [Figure 6] 10 is a flowchart when the display control unit according to the first embodiment changes the display mode of the second image. FIG. [Figure 7] FIG. 10 is a block diagram showing functions of a display control system according to a second embodiment. [Figure 8] FIG. 10 is a diagram showing a second image displayed small on the display unit. [Figure 9] FIG. 10 is a diagram showing a second image in which a passenger is displayed in an emphasized manner. [Figure 10] FIG. 10 is a block diagram showing functions of a display control system according to a fourth embodiment. [Figure 11] FIG. 10 is a diagram showing a state in which passengers included in the second image are displayed as icons on the display unit. [Figure 12] FIG. 10 is a diagram showing another aspect of the icon. [Figure 13] FIG. 10 is a block diagram for explaining the function of the digital mirror 1 according to the sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0019] In the following description, each component of each device represents a functional block, not a hardware configuration. Each component of each device is realized by any combination of hardware and software, centered around the CPU of any computer, memory, a program loaded into the memory, a storage medium such as a hard disk that stores the program, and a network connection interface. There are many variations in the realization method and device.

[0020] (First embodiment) Fig. 1 is a block diagram showing the functions of a display control system 100 according to the first embodiment. Fig. 2 is a diagram showing a vehicle 5 that realizes the display control system 100 according to the first embodiment. The display control system 100 according to the first embodiment will be described with reference to Figs. 1 and 2.

[0021] The display control system 100 includes a display unit 10, a first image capturing unit 20, a second image capturing unit 30, and a server 40. In the first embodiment, the server 40 is provided outside the moving body. The server 40 may be a so-called cloud server. In the first embodiment, the moving body is a vehicle 5.

[0022] (First camera unit 20) As shown in FIG. 2, in the first embodiment, the first photographing unit 20 is provided outside the vehicle. The first photographing unit 20 is provided at the rear of the vehicle 5. The first photographing unit 20 may be provided inside the vehicle and facing the outside of the vehicle. The inside of the vehicle (inside the vehicle) is the area where the driver and passengers are present. The outside of the vehicle is the area outside the body of the vehicle where the driver and passengers are not present.

[0023] The first photographing unit 20 photographs the area behind the vehicle 5. The first photographing unit 20 photographs a predetermined range θ1 centered around the first photographing unit 20. The first photographing unit 20 may be movable and may be able to swing around an axis in the height direction of the vehicle 5.

[0024] The first photographing unit 20 generates a first image. The first image is an image of the rear of the vehicle photographed by the first photographing unit 20. The first image is an image generated by photographing the rear of the vehicle 5. The first image is an image generated using the first photographing unit 20, and includes at least one of a still image and a video.

[0025] 1, the first image capturing unit 20 is configured to be able to communicate with the server 40 via a communication network 101. The communication network 101 includes, for example, communication over a 4G or 5G line. The first image capturing unit 20 is configured to include an external vehicle communication unit (not shown) for configuring the communication network 101. The first image capturing unit 20 transmits information related to the first image to the server 40.

[0026] (Second imaging unit 30) 2, the second photographing unit 30 is provided inside the vehicle. The second photographing unit 30 may be brought in from outside the vehicle 5 or may be attached to the vehicle 5 in advance. The second photographing unit 30 is provided, for example, on the ceiling of the vehicle 5. The second photographing unit 30 is provided, for example, on the ceiling between the driver's seat 5a and the passenger seat 5b.

[0027] The second photographing unit 30 photographs the interior of the vehicle. The second photographing unit 30 photographs a predetermined range θ2 centered on the second photographing unit 30. For example, the second photographing unit 30 photographs the interior of the vehicle so that the rear seat 5c of the vehicle 5 is photographed in its entirety. For example, the second photographing unit 30 photographs a passenger sitting in the rear seat 5c. The second photographing unit 30 may be movable and may be able to swing about an axis in the height direction of the vehicle 5.

[0028] The second photographing unit 30 generates a second image. The second image is an image of the interior of the vehicle photographed by the second photographing unit 30. The second image is an image generated by photographing the interior of the vehicle 5. The second image is an image generated using the second photographing unit 30, and includes at least one of a still image and a video.

[0029] 1, the second image capturing unit 30 is configured to be able to communicate with the server 40 via a communication network 101. The second image capturing unit 30 is configured to include an exterior vehicle communication unit (not shown) for configuring the communication network 101. The second image capturing unit 30 transmits information related to the second image to the server 40.

[0030] (Display section 10) In the first embodiment, the display unit 10 is a so-called electronic mirror. At least one of a first image and a second image is displayed on the display unit 10. As shown in FIG. 2, the display unit 10 is located at a position visible to the driver of the vehicle 5. The display unit 10 may be provided, for example, on the ceiling between the driver's seat 5a and the passenger seat 5b. The display unit 10 may be provided, for example, at a position closer to the driver's seat 5a than to the passenger seat 5b. The display unit 10 may be provided, for example, at a position closer to the driver's seat 5a than to the rear seat 5c.

[0031] 1, the display unit 10 includes a speaker as an audio output unit 11 and a microphone as an audio input unit 12. Note that a vibrator may be provided inside the display unit 10 (=digital mirror), and the speaker and microphone of the display unit 10 may be realized using the vibrator.

[0032] A microphone (not shown) may be provided near the rear seat 5c, and audio input to the microphone may be output from the audio output unit 11. A speaker (not shown) may be provided near the rear seat 5c, and the driver's audio input to the audio input unit 12 may be output from the speaker. In this way, the vehicle may be configured to enable communication between the driver and the passenger in the rear seat 5c via the display unit 10 (=digital mirror).

[0033] 1, the display unit 10 is configured to be able to communicate with a server 40 via a communication network 101. The display unit 10 is configured to include an external vehicle communication unit (not shown) for configuring the communication network 101.

[0034] (Display control unit 41) The server 40 includes a display control unit 41. The display control unit 41 displays at least one of a first image generated using the first photographing unit 20 and a second image generated using the second photographing unit 30 on the display unit 10. The display control unit 41 may display only the first image on the display unit 10, or may display only the second image on the display unit 10.

[0035] The display control unit 41 changes the display mode of at least one of the first image and the second image depending on a predetermined state, and displays both the first image and the second image on the display unit 10 when the predetermined state satisfies a first condition.

[0036] (predetermined state) The predetermined state includes at least one of the traveling state of the vehicle 5, the state behind the vehicle 5, the state of the passengers in the vehicle 5, and the state of the driver.

[0037] (Vehicle 5's driving status) The traveling state of the vehicle 5 includes at least one of the speed state of the vehicle 5 and the state of the road on which the vehicle 5 is traveling. The speed state of the vehicle 5 includes either one of a state related to the traveling speed of the vehicle 5 and a state related to whether the speed state of the vehicle 5 is stable. The state of the road on which the vehicle 5 is traveling includes at least one of a state related to whether the vehicle 5 is traveling on an ordinary road, whether the vehicle 5 is traveling on an expressway, and what the speed limit of the road on which the vehicle 5 is traveling is, for example.

[0038] The running state of the vehicle 5 also includes, for example, a state regarding whether the vehicle 5 is meandering. The running state of the vehicle 5 (= the speed state of the vehicle 5) also includes, for example, a state regarding whether the vehicle 5 is stopped, whether the vehicle 5 is caught in traffic, etc.

[0039] (Rear view of vehicle 5) The state behind the vehicle 5 includes, for example, a state regarding whether or not there is another vehicle behind the vehicle 5. The state behind the vehicle 5 includes, for example, a state regarding whether or not the distance between the vehicle 5 and the vehicle behind it is within a predetermined distance. Whether or not there is another vehicle behind the vehicle 5 and whether or not the distance between the vehicle 5 and the vehicle behind it is within a predetermined distance may be determined using at least one of the first image and vehicle traffic information (position information).

[0040] (Condition of passengers in vehicle 5) The state of the passenger in the vehicle 5 includes, for example, at least one of a state regarding whether the passenger is speaking, a state regarding the passenger's posture (whether the passenger is sleeping, awake, etc.), a state regarding the passenger's facial expression (whether the passenger is smiling, sad, etc.), and a state regarding the passenger's desire (whether the passenger wants to go to the toilet, whether the passenger is hungry, etc.). For example, the state of the passenger in the vehicle 5 is identified by analyzing the second image.

[0041] (Driver status) The driver's state includes, for example, at least one of a state relating to the degree of attention of the driver to the electronic mirror (display unit 10), a state relating to the driver's fatigue, and a state relating to the driver's physical condition.

[0042] 3 is a diagram showing how both the first image 21 and the second image 31 are displayed on the display unit 10. The display control unit 41 displays both the first image 21 and the second image 31 on the display unit 10 when a predetermined state satisfies a first condition.

[0043] The predetermined state satisfying the first condition includes at least one of the following: when the engine of the vehicle 5 is started, when the passenger speaks, and when the passenger does not speak for a predetermined period of time (for example, 5 minutes). Whether the passenger is speaking may be determined by performing image analysis using the second image, or may be determined using voice information of the passenger input to a microphone.

[0044] (Hardware configuration example) 4 is a diagram showing an example of the hardware configuration of the server 40. The server 40 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0045] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0046] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0047] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0048] The storage device 1040 is an auxiliary storage device realized by removable media such as a hard disk drive (HDD), a solid state drive (SSD), or a memory card, or a read-only memory (ROM), and has a recording medium. The recording medium of the storage device 1040 stores program modules that realize each function of the server 40 (for example, the display control unit 41). The processor 1020 loads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 may also function as a memory unit (not shown) connected to the server 40.

[0049] The input / output interface 1050 is an interface for connecting the server 40 with various input / output devices.

[0050] The network interface 1060 is an interface for connecting the server 40 to a network. This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The network interface 1060 may be connected to the network wirelessly or by wire. The server 40 may communicate with the display unit 10, the first image capturing unit 20, and the second image capturing unit 30 via the network interface 1060.

[0051] Note that, for example, the communication and control related parts of the display unit 10, the first image capturing unit 20, and the second image capturing unit 30 may also be realized by the same hardware configuration as the server 40.

[0052] (Operation example 1 of the first embodiment) FIG. 5 is a flow diagram when the display control unit 41 according to the first embodiment causes the display unit 10 to display both the first image 21 and the second image 31.

[0053] In step S10, the display control unit 41 (=server 40) acquires information about the first image from the first photographing unit 20.

[0054] In step S20, the display control unit 41 (=server 40) acquires information about the second image from the second photographing unit 30.

[0055] In step S30, the display control unit 41 determines whether the passenger in the rear seat 5c has not spoken for a predetermined time. The display control unit 41 may determine whether the passenger in the rear seat 5c has spoken by using voice information input to a microphone (not shown) mounted near the rear seat 5c. Alternatively, the display control unit 41 may determine whether the passenger in the rear seat 5c has spoken by using the second image.

[0056] If the display control unit 41 determines that the passenger in the rear seat 5c has not spoken within the predetermined time (YES in step S30), the process proceeds to step S40. If the display control unit 41 determines that the passenger in the rear seat has spoken within the predetermined time (NO in step S30), the process returns to step S30 again, and the process of step S30 is repeated.

[0057] Then, in step S40, the display control unit 41 causes both the first image 21 and the second image 31 to be displayed on the display unit 10. Thereafter, the process ends.

[0058] In step S30, instead of determining whether the passenger in the rear seat 5c has not spoken for a predetermined period of time, the display control unit 41 may determine whether the engine of the vehicle 5 has started or whether the passenger has spoken.

[0059] (Operation example 2 of the first embodiment) 6 is a flow diagram when the display control unit 41 according to the first embodiment changes the display mode of the second image 31. Operation example 2 is obtained by adding steps S50 and S60 after the flow of operation example 1.

[0060] In step S50 after step S40, the display control unit 41 determines whether the speed state of the vehicle 5 is stable. Whether the speed state of the vehicle is stable refers to a state in which the driver is driving within a predetermined speed range for a certain period of time. For example, if the driver is driving within a range of 55 km / h to 65 km / h for 10 minutes continuously, the speed state of the vehicle is said to be stable. Alternatively, whether the speed state of the vehicle is stable may be determined by whether the driver is driving within a predetermined acceleration range for a certain period of time.

[0061] If the display control unit 41 determines that the vehicle speed state is stable (YES in step S50), the process proceeds to step S60. If the display control unit 41 determines that the vehicle speed state is not stable (NO in step S50), the process ends. Note that if NO in step S50, the process may return to step S50 again and repeat the process of step S50.

[0062] In step S60, the display control unit 41 changes the display mode of the second image 31 so that the size of the second image 31 is larger than the size of the second image 31 when it was displayed in step S40. Note that steps S50 and S60 do not have to be performed after step S40, and may be performed after steps S10 and S20, for example.

[0063] As described above, according to the first embodiment, the display control system 100 includes a first photographing unit 20, a second photographing unit 30, a display unit 10, and a display control unit 41, and the display control unit 41 displays both the first image 21 and the second image 31 on the display unit 10 when a predetermined state satisfies a first condition.

[0064] By displaying both the first image 21 and the second image 31 on the display unit 10 when the predetermined state satisfies the first condition, the driver can grasp the situation behind the vehicle 5 and the status of the passengers inside the vehicle 5 at a glance on the display unit 10. Furthermore, the driver can grasp the status of the passengers inside the vehicle 5 (in the rear seats) without having to bother looking back or calling out to the passengers in the rear seats, thereby reducing the inconvenience. Therefore, the driver can easily grasp the status inside the vehicle 5 while checking (at the same time as checking) the situation behind the vehicle 5.

[0065] Furthermore, the display control unit 41 changes the display mode of at least one of the first image 21 and the second image 31 in accordance with a predetermined state. This allows the first image 21 and the second image 31 to be effectively displayed on the display unit 10 in accordance with various states (predetermined states).

[0066] Furthermore, the predetermined state satisfying the first condition includes at least one of the following: when the engine of the vehicle 5 is started; when the passenger speaks; and when the passenger does not speak for a predetermined period of time. For example, if the passenger does not speak for a predetermined period of time, the driver may become worried about whether the passenger is feeling unwell. By displaying both the first image 21 and the second image 31 on the display unit 10 when the passenger has not spoken for a predetermined period of time (when the predetermined state satisfies the first condition), the driver can visually check the passenger's condition and easily understand the passenger's condition. This allows the driver to concentrate on driving with peace of mind.

[0067] The predetermined state includes at least one of the traveling state of the vehicle 5, the state behind the vehicle 5, the state of the passengers in the vehicle 5, and the state of the driver. This allows the first image 21 and the second image 31 to be effectively displayed on the display unit 10 according to various situations.

[0068] Furthermore, the traveling state of the vehicle 5 includes at least one of the speed state of the vehicle 5 and the state of the road on which the vehicle 5 is traveling. This allows the first image 21 and the second image 31 to be effectively displayed on the display unit 10 according to various conditions related to the speed of the vehicle 5 and the road on which the vehicle 5 is traveling.

[0069] The display unit 10 also includes a speaker and a microphone. This allows the driver to communicate with passengers in the rear seats via the display unit 10. Furthermore, by installing a speaker and a microphone in the display unit 10 (electronic mirror) located in the driver's line of sight, the directionality of the driver's speech can be improved. This allows the driver and passengers in the rear seats to communicate effectively. Furthermore, by using a speaker separate from the existing speaker, the driver and passengers can communicate without interfering with the existing speaker system.

[0070] (Second embodiment) 7 is a block diagram showing functions of a display control system 100 according to the second embodiment. The server 40 according to the second embodiment further includes an inter-vehicle distance acquisition unit 42, unlike the server 40 according to the first embodiment.

[0071] The inter-vehicle distance acquisition unit 42 acquires inter-vehicle distance information relating to the distance (inter-vehicle distance) between the vehicle 5 and a vehicle behind the vehicle 5. The inter-vehicle distance acquisition unit 42 may acquire the inter-vehicle distance information through the first imaging unit 20, or may acquire the inter-vehicle distance information using vehicle position information or traffic information. In the predetermined state according to the second embodiment, the state behind the vehicle 5 includes a state in which the inter-vehicle distance between the vehicle 5 and the vehicle behind the vehicle 5 is equal to or shorter than a predetermined distance.

[0072] 8 is a diagram showing the second image 31 displayed small on the display unit 10. When the inter-vehicle distance becomes equal to or less than a predetermined distance, the display control unit 41 changes the display mode of the second image 31 so that the size of the second image 31 displayed on the display unit 10 is reduced or so that the second image 31 is not displayed at all. The second image 31 in FIG. 8 is displayed small on the display unit 10 compared to the second image 31 in FIG. 3.

[0073] When the distance between vehicle 5 and the vehicle behind vehicle 5 is small (including when the vehicle is being tailgated), the driver must concentrate on driving. By displaying second image 31 in a small size or not displaying second image 31 at all, the driver's concentration on second image 31 can be reduced. This improves the driver's concentration on driving.

[0074] Furthermore, in another embodiment, the inter-vehicle distance acquisition unit 42 may acquire rear vehicle information. The rear vehicle information is information relating to the presence or absence of a vehicle behind the vehicle 5. The inter-vehicle distance acquisition unit 42 may acquire the rear vehicle information through the first photographing unit 20, or may acquire the rear vehicle information using vehicle position information or traffic information. In the predetermined state according to the second embodiment, the state behind the vehicle 5 includes a state in which there is no vehicle behind the vehicle 5.

[0075] When there is no vehicle behind the vehicle 5, the display control unit 41 changes the display mode of the second image 31 so that the size of the second image 31 displayed on the display unit 10 is increased. When there is no vehicle behind the vehicle 5, there may be no problem in increasing the degree of concentration on the display unit 10 (= the second image 31). By displaying the second image 31 in a larger size when there is no vehicle behind the vehicle 5, the situation of the passengers in the rear seats can be effectively grasped.

[0076] The display control unit 41 according to the second embodiment may output a sound effect in accordance with a predetermined state. The display control unit 41 may output a warning sound or a warning mark to the display unit 10. For example, when the distance between the vehicle 5 and a vehicle behind the vehicle becomes equal to or shorter than a predetermined distance, the display control unit 41 may output a warning sound from a speaker. This makes it easier for the driver to notice that the distance between the vehicle behind the vehicle 5 and the vehicle behind is getting closer.

[0077] (Third embodiment) The third embodiment differs in function from the display control unit 41 of the first embodiment. The basic configuration is the same as that shown in FIG. 1. The state of the passenger in the vehicle 5 in a predetermined state according to the third embodiment includes a state in which the passenger is speaking. When the passenger is speaking, the display control unit 41 according to the third embodiment changes the display mode of the second image 31 so that the passenger 6 included in the second image 31 is displayed in an emphasized manner.

[0078] Fig. 9 is a diagram showing the second image 31 in which the passenger 6a is displayed with emphasis. As shown in Fig. 9, the display control unit 41 may cause the display unit 10 to display the passenger 6a, of the multiple passengers 6, with the outer frame 7 of the passenger 6a who is speaking in a thicker shape. The display control unit 41 may also display an emphasis icon 8 above the passenger 6a who is speaking to emphasize the passenger 6a. Although not shown, the display control unit 41 may also color the outer frame of the passenger 6a (for example, red) to emphasize the passenger 6a.

[0079] In this way, the display control unit 41 highlights the speaking passenger 6a and displays it in the second image 31, allowing the driver to immediately recognize which of the multiple passengers 6 is speaking.

[0080] (Fourth embodiment) 10 is a block diagram showing functions of a display control system 100 according to the fourth embodiment. Unlike the first embodiment, the server 40 according to the fourth embodiment further includes a driver information acquisition unit 43. The driver information acquisition unit 43 acquires driver status information. The driver status information includes information relating to whether or not the driver's attention is declining.

[0081] The driver information acquisition unit 43 may acquire the driver state information by analyzing images (for example, analyzing the driver's line of sight) using a camera (or the second imaging unit 30) mounted in the vehicle 5. The driver information acquisition unit 43 may acquire the driver state information by using, for example, vital information of the driver (blood pressure, pulse rate, amount of sweat, etc.) obtained from in-vehicle equipment (steering wheel, seat belt, etc.) or a device worn by the driver.

[0082] The driver's state in the predetermined state according to the fourth embodiment includes a state in which the driver's attention is reduced. When the driver's attention is reduced, the display control unit 41 according to the fourth embodiment changes the display mode of the second image so that the second image is not displayed. This reduces the driver's concentration on the second image when the driver's attention is reduced, and allows the driver to focus on the first image (an image behind the vehicle). This can reduce traffic accidents.

[0083] (Fifth embodiment) FIG. 11 is a diagram showing how the passenger 6 included in the second image 31 is displayed in the form of an icon on the display unit 10. The display control unit 41 according to the fifth embodiment displays the passenger 6 included in the second image 31 in the form of an icon on the display unit 10. The icons 44 represent, for example, the emotions of the passenger 6, such as joy, anger, sadness, and happiness. The display control unit 41 switches the actual image of the passenger 6 (the second image 31) to the icon 44, for example, while driving. The display control unit 41 switches, for example, the image of the passenger 6 (see area A in FIG. 11) to at least one of the icons 44. The display control unit 41 may switch only the face of the passenger 6 in the image of the passenger 6 to the icon 44, or may switch the entire passenger 6 to the icon 44.

[0084] The display control unit 41 changes the icon state depending on the state of the passenger 6. The state of the passenger includes, for example, a state in which the passenger is happy, a state in which the passenger is angry, a state in which the passenger is sad, and a state in which the passenger is having fun. For example, when the passenger 6 is crying, the display control unit 41 may change the icon state to an icon 44a that represents a sad emotion.

[0085] 12 is a diagram showing another aspect of the icon 44. The icon 44 may represent the passenger's requests in addition to the above-mentioned states of joy, anger, sadness, and happiness. The state of the passenger 6 includes, in addition to the above-mentioned states of joy, anger, sadness, and happiness, a state in which the passenger 6 wants to go to the toilet, a state in which the passenger is hungry, a state in which the passenger is hot, a state in which the passenger is cold, and the like.

[0086] For example, icon 44w indicates that the passenger 6 wants to go to the toilet, and icon 44x indicates that the passenger 6 is hungry. Note that icons indicating the passenger 6's needs (for example, toilet icon 45a and meal icon 45b) may be displayed simultaneously with the icons representing facial expressions.

[0087] The predetermined state according to the fifth embodiment includes a traveling state of the vehicle 5. The traveling state of the vehicle 5 includes a state in which the vehicle 5 is stopped. When the vehicle 5 is stopped, the display control unit 41 changes the display mode of the second image 31 so that the passenger 6 displayed in icon form is displayed as an actual image. The display control unit 41 may determine whether the vehicle 5 is stopped based on whether the parking brake is functioning, or may detect the speed of the vehicle to determine whether the vehicle is stopped.

[0088] As described above, according to the fifth embodiment, the display control unit 41 displays the passenger 6 included in the second image 31 as an icon on the display unit 10. This makes it possible to reduce the driver's concentration on the display unit 10. As a result, it is possible to reduce the occurrence of traffic accidents.

[0089] Furthermore, the display control unit 41 changes the icon appearance depending on the state of the passenger 6. This allows the driver to easily understand the state of the passenger 6.

[0090] Furthermore, when the vehicle 5 is stopped, the display control unit 41 changes the display mode of the second image 31 so that the passenger 6 displayed in icon form is displayed as an actual image. When the vehicle 5 is stopped, there is no problem if the driver concentrates on looking at the display unit 10 (second image 31). By providing the above configuration, the driver can safely grasp the condition of the passenger 6.

[0091] (Sixth embodiment) FIG. 13 is a block diagram illustrating the function of the electronic mirror 1 according to the sixth embodiment. The electronic mirror 1 according to the sixth embodiment includes a display unit 10 and a display control unit 41. As in the first embodiment, the display unit 10 is disposed in a position visible to the driver of the vehicle 5. In the sixth embodiment, the display unit 10 includes a display portion of the electronic mirror 1. As in the first embodiment, the display control unit 41 displays, on the display unit 10, at least one of a first image generated using the first image capture unit 20 and a second image generated using the second image capture unit 30. As in the first embodiment, the display control unit 41 changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit 10 when the predetermined state satisfies a first condition.

[0092] The display control unit 41 may be configured to be mounted inside the display unit 10 or may be configured to be mounted outside the display unit 10. When the display control unit 41 is configured to be mounted outside the display unit 10, the display control unit 41 is provided inside the vehicle and does not communicate with a server or the like outside the vehicle. The display control unit 41 communicates with the display unit 10, the first photographing unit 20, the second photographing unit 30, and the like mounted on the vehicle.

[0093] As described above, according to the sixth embodiment, the digital mirror 1 can be realized without using the server 40.

[0094] In the sixth embodiment, the first photographing unit 20 photographs the rear of the vehicle 5, and the second photographing unit 30 photographs the interior of the vehicle 5. This makes it possible to realize an electronic mirror 1 that allows the driver to easily grasp the situation inside the vehicle while checking the situation behind the vehicle.

[0095] (Variation) As a modified example, the display control unit 41 may detect that the parking brake is functioning and display only the second image on the display unit 10.

[0096] Furthermore, when the vehicle is being tailgated (when the inter-vehicle distance repeatedly becomes smaller and larger), the display control unit 41 may display only the first image on the display unit 10.

[0097] Furthermore, for example, when there are multiple passengers 6 in the rear seats, the display control unit 41 may display the passengers 6 one by one on the display unit 10 (second image 31) rather than displaying all of the passengers 6 at the same time. In this case, the display control unit 41 may switch the display of the passengers 6 for each speaker.

[0098] Although the embodiments have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0099] The display control unit 41 according to the first embodiment may be included in the display unit 10. In this case, the display control system 100 can be realized without the server 40.

[0100] The display control system 100 is realized using one or more computers, such as the display unit 10, the first image capturing unit 20, the second image capturing unit 30, and the server 40. The number of computers used to realize the display control system 100 is arbitrary.

[0101] The display unit 10 according to the first embodiment may be a head-up display or any other display that can be placed in a position visible to the driver.

[0102] Below, examples of reference forms are given. 1. A first imaging unit that images the rear of a moving object; a second imaging unit that images the inside of the moving body; a display unit located at a position visible to a driver of the vehicle; a display control unit that displays at least one of a first image generated using the first imaging unit and a second image generated using the second imaging unit on the display unit, The display control unit changes the display mode of at least one of the first image and the second image in accordance with a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition. 2. In the display control system described in 1., A display control system, wherein when the specified state satisfies the first condition, the state includes at least one of when the engine of the vehicle starts, when the passenger speaks, and when the passenger does not speak for a specified period of time. 3. In the display control system according to 1. or 2., The predetermined state includes at least one of a traveling state of the moving body, a state behind the moving body, a state of a passenger in the moving body, and a state of the driver. 4. In the display control system described in 3., A display control system, wherein the traveling state of the moving object includes at least one of the speed state of the moving object and the state of the road on which the moving object is traveling. 5. In the display control system according to 3. or 4., The state behind the moving body includes a state in which the inter-vehicle distance between the moving body and a vehicle behind the moving body is equal to or shorter than a predetermined distance, A display control system in which, when the inter-vehicle distance becomes less than a predetermined value, the display control unit changes the display mode of the second image displayed on the display unit so that the size of the second image is reduced or so that the second image is not displayed at all. 6. In the display control system according to any one of 3. to 5., the state behind the moving object includes a state where there is no vehicle behind the moving object, A display control system in which, when there is no vehicle behind the moving body, the display control unit changes the display mode of the second image so that the size of the second image displayed on the display unit is larger. 7. In the display control system according to any one of 3. to 6., the state of the passenger in the vehicle includes a state in which the passenger is speaking, A display control system in which, when the passenger is speaking, the display control unit changes the display mode of the second image so that the passenger included in the second image is displayed in an emphasized manner. 8. In the display control system according to any one of 3. to 7., the driver's state includes a state in which the driver's attention is reduced, When the driver's attention is reduced, the display control unit changes the display mode of the second image so that the second image is not displayed. 9. In the display control system according to any one of 1. to 8., The display control system is configured so that the display control unit displays the passengers included in the second image in the form of icons on the display unit. 10. In the display control system according to 9., The display control system is configured to change the icon appearance depending on the state of the passenger. 11. In the display control system according to 9. or 10., the predetermined state includes a running state of the moving body, the traveling state of the moving body includes a state in which the moving body is stopped, A display control system in which, when the moving body is stopped, the display control unit changes the display mode of the second image so that the passenger displayed in icon form is displayed as an actual image. 12. In the display control system according to any one of 1. to 11., The display control system, wherein the display control unit outputs a sound effect in accordance with the predetermined state. 13. In the display control system according to any one of 1. to 12., The display control system, wherein the display unit includes a speaker and a microphone. 14. A display unit located in a position visible to the driver of the vehicle; a display control unit that displays at least one of a first image generated using a first imaging unit and a second image generated using a second imaging unit on the display unit, The display control unit changes the display mode of at least one of the first image and the second image in accordance with a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition. 15. In the electronic mirror of 14., the first photographing unit photographs a rear view of the moving object, The second photographing unit is an electronic mirror that photographs the inside of the moving body. 16. One or more computers that implement a display control system Photograph the rear of the moving object, Taking an image of the inside of the moving body, displaying at least one of a first image generated by photographing the rear of the moving body and a second image generated by photographing the inside of the moving body on a display unit located at a position visible to a driver of the moving body; A control method for a display control system, which changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition. 17. One or more computers that implement a display control system, Procedure for photographing the rear of a moving object, a step of photographing the inside of the moving body; a step of displaying at least one of a first image generated by photographing the rear of the moving body and a second image generated by photographing the inside of the moving body on a display unit located at a position visible to a driver of the moving body; A program for executing a procedure for changing the display mode of at least one of the first image and the second image in accordance with a predetermined state, and displaying both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

[0103] This application claims priority based on Japanese Patent Application No. 2023-018901, filed February 10, 2023, the disclosure of which is incorporated herein by reference in its entirety. [Explanation of symbols]

[0104] 1. Electronic mirror 5 vehicles 5a Driver's seat 5b passenger seat 5c rear seat 6 Passengers 10 Display section 20 First Filming Section 21 First image 30 2nd Filming Department 31 Second image 40 servers 41 Display control unit 42 Inter-vehicle distance acquisition section 43 Driver information acquisition unit 44 icons 100 Display Control System

Claims

1. a first imaging unit that images a rear view of the moving object; a second imaging unit that images the inside of the moving object; a display unit located at a position visible to a driver of the vehicle; a display control unit that displays at least one of a first image generated using the first imaging unit and a second image generated using the second imaging unit on the display unit, A display control system in which the display control unit changes the display mode of at least one of the first image and the second image in accordance with a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

2. The display control system according to claim 1, A display control system, wherein when the specified state satisfies the first condition, the state includes at least one of when the engine of the vehicle starts, when a passenger in the vehicle speaks, and when the passenger does not speak for a specified period of time.

3. 3. The display control system according to claim 1, The predetermined state includes at least one of a traveling state of the moving body, a state behind the moving body, a state of a passenger in the moving body, and a state of the driver.

4. 4. The display control system according to claim 3, A display control system, wherein the traveling state of the moving object includes at least one of the speed state of the moving object and the state of the road on which the moving object is traveling.

5. 4. The display control system according to claim 3, The state behind the moving body includes a state in which the inter-vehicle distance between the moving body and a vehicle behind the moving body is equal to or shorter than a predetermined distance, A display control system in which, when the inter-vehicle distance becomes less than a predetermined value, the display control unit changes the display mode of the second image displayed on the display unit so that the size of the second image is reduced or so that the second image is not displayed.

6. 4. The display control system according to claim 3, the state behind the moving object includes a state where there is no vehicle behind the moving object, A display control system in which, when there is no vehicle behind the moving body, the display control unit changes the display mode of the second image so that the size of the second image displayed on the display unit is larger.

7. 4. The display control system according to claim 3, the state of the passenger in the vehicle includes a state in which the passenger is speaking, A display control system in which, when the occupant is speaking, the display control unit changes the display mode of the second image so that the occupant included in the second image is displayed in an emphasized manner.

8. 4. The display control system according to claim 3, the driver's state includes a state in which the driver's attention is reduced, When the driver's attention is reduced, the display control unit changes the display mode of the second image so that the second image is not displayed.

9. 3. The display control system according to claim 1, The display control system, wherein the display control unit displays the passenger included in the second image on the display unit in the form of an icon.

10. 10. The display control system according to claim 9, The display control system is configured to change the icon appearance depending on the state of the passenger.

11. 10. The display control system according to claim 9, the predetermined state includes a running state of the moving body, the traveling state of the moving body includes a state in which the moving body is stopped, A display control system in which, when the moving body is stopped, the display control unit changes the display mode of the second image so that the passenger displayed in icon form is displayed as an actual image.

12. 3. The display control system according to claim 1, The display control system, wherein the display control unit outputs a sound effect in accordance with the predetermined state.

13. 3. The display control system according to claim 1, The display control system, wherein the display unit includes a speaker and a microphone.

14. a display unit located at a position visible to a driver of the vehicle; a display control unit that displays at least one of a first image generated using the first imaging unit and a second image generated using the second imaging unit on the display unit, The display control unit changes the display mode of at least one of the first image and the second image depending on a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

15. The electronic mirror of claim 14, the first photographing unit photographs a rear view of the moving object, The second photographing unit is an electronic mirror that photographs the inside of the moving body.

16. One or more computers that implement a display control system Photograph the rear of the moving object, Taking an image of the inside of the moving body, displaying at least one of a first image generated by photographing the rear of the moving body and a second image generated by photographing the inside of the moving body on a display unit located at a position visible to a driver of the moving body; A control method for a display control system, which changes the display mode of at least one of the first image and the second image according to a predetermined state, and displays both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

17. One or more computers that implement the display control system Procedure for photographing the rear of a moving object, a step of photographing the inside of the moving body; a step of displaying at least one of a first image generated by capturing an image of the rear of the moving body and a second image generated by capturing an image of the inside of the moving body on a display unit located at a position visible to a driver of the moving body; A program for executing a procedure for changing the display mode of at least one of the first image and the second image in accordance with a predetermined state, and displaying both the first image and the second image on the display unit when the predetermined state satisfies a first condition.

Images