Display control device and display control method

Abstract

To allow a user such as a driver to easily distinguish an image of the outside of a vehicle from an image of the inside of the vehicle in a display control device which causes a display device to display the image of the outside of the vehicle and the image of the inside of the vehicle.SOLUTION: A display control device comprises: a first imaging unit which generates a first photographed image obtained by imaging the outside of a vehicle; a second imaging unit which generates a second photographed image obtained by imaging the inside of the vehicle; and a display control unit which displays the first photographed image and the second photographed image on a display unit. The display control unit corrects a partial or the entire display mode of at least one of the first photographed image and the second photographed image to display the images on the display unit when simultaneously displaying the first photographed image and the second photographed image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a display control device and a display control method.

Background Art

[0002] There is known a display control technique in which an image of the rear of a vehicle is captured by a camera mounted on the vehicle, and the captured rear image is displayed on a display device or the like, causing the display device or the like to function as an electronic mirror.

[0003] Also, there is known a technique in which an image of a passenger in the rear seat is captured by a camera mounted on the vehicle, and the captured passenger image is displayed on a display device or the like together with an image of the rear of the vehicle (see, for example, Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the technique disclosed in Patent Document 1, an image of the rear seat is displayed in the image of the rear of the vehicle displayed on a display device or the like. In this case, since the image of the rear seat also includes the image of the rear of the vehicle, there is a situation where there is a landscape within a landscape, and there is a problem that it is difficult for a user such as a driver to distinguish between the image of the rear of the vehicle and the image of the rear seat.

[0006] In the technology disclosed in Patent Document 2, a composite image obtained by compositing a subject image obtained by removing a background image from an image of a rear seat onto an image behind the vehicle is displayed on a display device or the like. In this case, although the scenery behind the vehicle can be excluded from the image of the rear seat, there is a problem that it is difficult to identify whether the person shown in the composite image is inside or outside the vehicle.

[0007] As described above, in the conventional technology, in an image display device that displays an image outside the vehicle and an image inside the vehicle on a display device, it has been difficult for a user such as a driver to distinguish between the image outside the vehicle and the image inside the vehicle.

[0008] One embodiment of the present invention has been made in view of the above problems, and in a display control device that displays an image outside the vehicle and an image inside the vehicle on a display device, enables a user such as a driver to easily distinguish between the image outside the vehicle and the image inside the vehicle.

Means for Solving the Problems

[0009] In order to solve the above problems, a display control device according to an embodiment of the present invention includes Rear of the vehicle a first imaging unit that generates a first captured image obtained by imaging Occupant in the rear seat a second imaging unit that generates a second captured image obtained by imaging, and a display control unit that displays the first captured image and the second captured image on a display unit. When the display control unit simultaneously displays the first captured image and the second captured image, Display the second captured image within the range of the first captured image displayed on the display unit, change the color scheme of the background of the occupant in the second captured image, and make the background of the occupant with the changed color scheme transparent so that the first captured image can be visually recognized.

Effects of the Invention

[0010] According to one embodiment of the present invention, in a display control device that displays an image outside the vehicle and an image inside the vehicle on a display device, a user such as a driver can easily distinguish between the image outside the vehicle and the image inside the vehicle.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

MODE FOR CARRYING OUT THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0013] <Configuration example of a display control system> FIG. 1 is a diagram showing a configuration example of a display control system according to an embodiment. The display control system 1 is a system that causes a display device 13 to function as an electronic mirror by displaying, on the display device 13, a first captured image obtained by capturing the rear of a vehicle 10 and a second captured image obtained by capturing an occupant inside the vehicle 10, for example, in the vehicle 10 such as an automobile. In the example of FIG. 1, the display control system 1 includes a display control device 100 mounted on the vehicle 10, an outside vehicle camera 11, an inside vehicle camera 12, a display device 13, and the like.

[0014] The display control device 100 is an electronic control device having a configuration of a computer such as an ECU (Electric Control Unit) mounted on the vehicle 10, for example. The display control device 100 realizes functional configurations such as a first imaging unit 120, a second imaging unit 130, and a display control unit 110 by executing a predetermined program with a computer included in the display control device 100. Note that at least a part of each of the above functional configurations may be realized by hardware.

[0015] The first imaging unit 120 executes a first imaging process of controlling the outside vehicle camera 11 that captures the outside of the vehicle 10 and generating a first captured image obtained by capturing the outside of the vehicle. Preferably, the first imaging unit 120 captures the rear of the vehicle 10 using the outside vehicle camera 11 and generates a first captured image obtained by horizontally flipping the captured image. Note that the process of horizontally flipping the captured image may be executed by the display control unit 110.

[0016] The second imaging unit 130 executes a second imaging process of controlling the inside vehicle camera 12 that captures the inside of the vehicle 10 and generating a second captured image obtained by capturing the inside of the vehicle. Preferably, the second imaging unit 130 captures an occupant in the rear seat of the vehicle 10 using the inside vehicle camera 12 and generates a second captured image obtained by horizontally flipping the captured image. Note that the process of horizontally flipping the captured image may be executed by the display control unit 110.

[0017] The display control unit 110 generates a display image that simultaneously displays the first captured image and the second captured image, and causes the display device 13 to display the generated display image, thereby executing a display control process for causing the display device 13 to function as an electronic mirror (room mirror).

[0018] The display device (display unit) 13 is provided, for example, at the same position as a conventional room mirror (indoor rearview mirror), and displays a display image according to the control from the display control unit 110. The display device 13 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display.

[0019] FIG. 2 is a diagram showing an image of a conventional room mirror. The conventional room mirror (indoor rearview mirror) 201 is provided, for example, at the upper center inside the front window 202 of the vehicle 10, and is used by a driver driving the vehicle to check the state of the rear 203 of the vehicle or the passenger 204 in the rear seat of the vehicle.

[0020] FIG. 3 is a diagram showing an image of an example of a display image that displays a rear image and a passenger image. The display device 13 is provided, for example, at the upper center inside the front window 202 of the vehicle 10, and displays a first captured image 301 of the rear of the vehicle 10 and a second captured image of the passenger in the rear seat of the vehicle 10.

[0021] In the example of FIG. 3, similar to the technique disclosed in Patent Document 2, a display image 303 is displayed on the display device 13, which is a composite of a first captured image 301 of the rear of the vehicle 10 and a passenger image 302 obtained by removing the background image from the second captured image of the passenger in the rear seat. In this case, there is a problem that it is difficult for a driver or the like driving the vehicle 10 to identify whether the person shown in the display image 303 is a person inside the vehicle 10 or a person outside the vehicle 10.

[0022] Also, like the technology disclosed in Patent Document 1, it is also possible to perform PinP (Picture in Picture) display of a second captured image of the occupant in the rear seat of the vehicle 10 in the first captured image 301 that captures the rear of the vehicle 10. However, in this case, since the second captured image of the occupant in the rear seat also includes an image of the rear of the vehicle 10, there is a situation where there is a scene within a scene, and there is a problem that it is difficult for the driver to distinguish between the image of the rear of the vehicle 10 and the image of the rear seat.

[0023] Therefore, as shown in FIG. 1, the display control unit 110 according to the present embodiment includes a display mode correction unit 111, a display image generation unit 112, and the like.

[0024] When the display control unit 110 simultaneously displays the first captured image and the second captured image on the display device 13, the display mode correction unit 111 executes a display mode correction process for correcting the display mode of at least a part or all of one of the first captured image and the second captured image. For example, when simultaneously displaying the first captured image and the second captured image, the display mode correction unit 111 executes a color scheme change process for changing the color scheme of the background of the occupant captured in the second captured image. In addition, the display mode correction unit 111 executes a transparency process for making the area with the changed color scheme transparent. For example, when the display mode correction unit 111 changes the color scheme of the background of the occupant captured in the second captured image, the background of the occupant with the changed color scheme is made transparent so that the first captured image can be visually recognized. Note that the display mode correction process may include various processes such as a blurring process and a mosaic process in addition to the color scheme change process and the transparency process.

[0025] For example, the display image generation unit 112 executes a display image generation process for generating a display image by synthesizing the first captured image and the second captured image on which the display mode correction unit 111 has executed a color scheme change process, a transparency process, and the like. In addition, the display image generation unit 112 displays the generated display image on the display device 13 by outputting the display image to the display device 13.

[0026] FIG. 4 is a diagram showing an image of a display image displayed by a display device according to an embodiment. This diagram shows an example image of a display image 401 when the display control device 100 according to the present embodiment simultaneously displays a first captured image 402 and a second captured image 403 on the display device 13. In the example of FIG. 4, the display control unit 110 displays, on the display device 13, a display image 401 in which a second captured image 403 of the rear seat of the vehicle 10 is displayed in PinP in the first captured image 402 of the rear of the vehicle 10.

[0027] Also, in the example of FIG. 4, the display control unit 110 changes the color scheme of the background 405 of the passenger 404 in the second captured image 403 using the display mode correction unit 111. For example, the display control unit 110 may fill the background 405 of the passenger 404 in the second captured image 403 with a predetermined color using the display mode correction unit 111. Thereby, the driver of the vehicle 10 can easily distinguish between the first captured image 402 of the rear of the vehicle 10 and the second captured image 403 of the rear seat of the vehicle 10.

[0028] Preferably, the display control unit 110 performs a transparency process on the background 405 of the passenger 404 in the second captured image 403 using the display mode correction unit 111. For example, the display control unit 110 makes the background 405 filled with a predetermined color transparent so that the first captured image 402 can be visually recognized using a known image processing technique. Thereby, the display control unit 110 can reduce the area in the first captured image 402 that cannot be visually recognized by the second captured image 403 in the display image 401 as shown in FIG. 4.

[0029] As described above, according to the present embodiment, in the display control device 100 that displays an image outside the vehicle and an image inside the vehicle on the display device 13, a user such as a driver can easily distinguish between the image outside the vehicle and the image inside the vehicle.

[0030] <Hardware Configuration> FIG. 5 is a diagram showing an example of the hardware configuration of a display control device according to an embodiment. The display control device 100 has a computer configuration and includes, for example, a CPU (Central Processing Unit) 501, a memory 502, a storage device 503, a GPU (Graphics Processing Unit) 504, a communication device 505, an I / F (Interface) 506, and a bus 507, etc.

[0031] The CPU 501 is a processor that realizes each functional configuration of the display control device 100 as shown in FIG. 1 by executing a predetermined program stored in a storage medium such as the storage device 503. The memory 502 includes, for example, a RAM (Random Access Memory) which is a volatile memory used as a work area of the CPU 501, and a ROM (Read Only Memory) which is a non-volatile memory storing a startup program of the CPU 501, etc. The storage device 503 is a non-volatile and large-capacity storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), for example.

[0032] The GPU 504 is an image processing processor that mainly executes various processes related to image processing faster than the CPU 501. The communication device 505 includes one or more communication interfaces for connecting the display control device 100 to a communication network. For example, the communication device 505 includes a communication interface for connecting the display control device 100 to the in-vehicle network of the vehicle 10. Also, the communication device 505 may include a wireless communication device or the like for connecting the display control device 100 to a WAN (Wide Area Network) such as the Internet, for example.

[0033] The I / F 506 is an interface for connecting external devices such as the outside vehicle camera 11, the inside vehicle camera 12, and the line-of-sight detection device 1101 (to be described later) to the display control device 100. The bus 507 is commonly connected to each of the above components and transmits, for example, address signals, data signals, and various control signals.

[0034] <Flow of processing> Subsequently, the flow of processing of the display control method according to the present embodiment will be described.

[0035] [First Embodiment] FIG. 6 is a flowchart showing an example of processing of the display control device according to the first embodiment. The display control device 100 repeatedly executes the processing shown in FIG. 6, for example, while the ignition switch of the vehicle 10 is on.

[0036] In step S601, the first imaging unit 120 controls the outside vehicle camera 11 to generate a first captured image that captures, for example, the rear of the vehicle 10. Preferably, the first imaging unit 120 generates a first captured image to be displayed on the display device 13 by horizontally flipping the image captured by the outside vehicle camera 11.

[0037] In step S602, the second imaging unit 130, in parallel with the processing of step S601, controls the inside vehicle camera 12 to generate a second captured image that captures, for example, the occupant in the rear seat of the vehicle 10. Preferably, the second imaging unit 130 generates a second captured image to be displayed on the display device 13 by horizontally flipping the image captured by the inside vehicle camera 12.

[0038] In step S603, the display control unit 110 determines whether or not an occupant is shown in the second captured image generated by the second imaging unit 130. If an occupant is shown in the second captured image, the display control unit 110 shifts the processing to step S604. On the other hand, if no occupant is shown in the second captured image, the display control unit 110 shifts the processing to step S606.

[0039] When shifting to step S604, the display control unit 110 changes the color scheme of the background of the passenger in the second captured image using the display mode correction unit 111. As an example, the display control unit 110 uses the display mode correction unit 111 to fill the background of the passenger in the second captured image with a predetermined color (for example, blue background, etc.).

[0040] In step S605, the display control unit 110 uses the display image generation unit 112 to generate a display image for simultaneously displaying the first captured image and the second captured image with the color scheme of the passenger changed.

[0041] On the other hand, when shifting from step S603 to step S606, the display control unit 110 uses the display image generation unit 112 to generate a display image for displaying the first captured image. Thereby, the display control unit 110 stops the display of the second captured image on the display device 13.

[0042] In step S607, the display control unit 110 outputs the display image generated by the display image generation unit 112 to the display device 13. Thereby, the display device 13 displays a display image 701 as shown in FIG. 7, for example.

[0043] FIG. 7 is a diagram showing an example of a display image according to the first embodiment. This diagram shows an image example of the display image 701 displayed by the display device 13 when there is a passenger in the rear seat of the vehicle 10.

[0044] In the example of FIG. 7, the display control device 100 displays on the display device 13 a display image 701 in which a second captured image 703 of the passenger in the rear seat of the vehicle 10 is PinP-displayed within a first captured image 702 of the rear of the vehicle 10. Also, in the second captured image 703, the background 705 of the passenger 704 is filled with a predetermined color. Therefore, the driver of the vehicle 10 can easily distinguish the passenger 704 sitting in the rear seat of the vehicle 10 from the person 706 outside the vehicle behind the vehicle 10.

[0045] Note that the display image 701 shown in FIG. 7 is an example. For example, the display control device 100 may display, on the display device 13, a display image in which a first captured image 702 capturing the rear of the vehicle 10 and a second captured image 703 capturing the passengers in the rear seat of the vehicle 10 are arranged side by side horizontally, for example.

[0046] As described above, according to the first embodiment, in the display control device 100 that causes the display device 13 to display an image outside the vehicle and an image inside the vehicle, a user such as a driver can easily distinguish between the image outside the vehicle and the image inside the vehicle.

[0047] Note that in step S604 of FIG. 6, instead of changing the color scheme of the background of the passengers in the second captured image using the display mode correction unit 111, the display control unit 110 may execute a process of blurring the background. Even in this case, the driver of the vehicle 10 can easily distinguish between the passenger 704 sitting in the rear seat of the vehicle 10 and the person 706 outside the vehicle behind the vehicle 10.

[0048] [Second Embodiment] In the second embodiment, an example in which the display control device 100 performs a transparency process on the background 705 of the passengers 704 in the second captured image 703 in the display image 701 as shown in FIG. 7 will be described.

[0049] FIG. 8 is a flowchart showing an example of the processing of the display control device according to the second embodiment. Note that, among the processes shown in FIG. 8, the processes of steps S601 to S604, S606, and S607 are the same as the processes of steps S601 to S604, S606, and S607 of the first embodiment, and thus the description thereof will be omitted here.

[0050] In step S801, the display control unit 110 uses the display mode correction unit 111 to execute a transparency process of making the background of the passengers in the second captured image transparent so that the first captured image can be captured.

[0051] In step S802, the display control unit 110 generates a display image for displaying the second captured image within the first captured image by using the display image generation unit 112.

[0052] Through the above processing, the display control device 100 can display, for example, a display image 401 as shown in FIG. 4 on the display device 13.

[0053] In the example of FIG. 4, the display control device 100 displays, on the display device 13, a display image 401 in which a second captured image 403 of a passenger in the rear seat of the vehicle 10 is PinP-displayed within the range of a first captured image 402 of the rear of the vehicle 10. Also, in the second captured image 703, the background 705 of the passenger 704 is filled with a predetermined color and is subjected to a transparency process so that the first captured image 402 can be visually recognized.

[0054] In the first embodiment, as shown in FIG. 7, the driver of the vehicle 10 cannot visually recognize the area in the first captured image 702 where the second captured image 703 is displayed. On the other hand, in the second embodiment, as shown in FIG. 4, the driver of the vehicle 10 can visually recognize the first captured image 402 in the background 405 of the passenger 404 in the area where the second captured image 403 is displayed.

[0055] Note that the processing shown in FIG. 8 is an example, and the display control device 100 may display a display image 401 as shown in FIG. 4 on the display device 13 according to, for example, another processing procedure as shown in FIG. 9.

[0056] FIG. 9 is a flowchart showing another example of the processing of the display control device according to the second embodiment. Note that, among the processing shown in FIG. 9, the processing of steps S601 to S603, S606, and S607 is the same as the processing of steps S601 to S603, S606, and S607 in the first embodiment, and thus the description thereof is omitted here.

[0057] In step S901, the display control unit 110 uses the display mode correction unit 111 to change the color scheme (color combination) of the display area 1002 for displaying the second captured image in the first captured image 1001, for example, as shown in FIG. 10.

[0058] In step S902, the display control unit 110 extracts an image of the occupant excluding the background image from the second captured image, for example, using the display image generation unit 112.

[0059] In step S903, the display control unit 110 uses the display image generation unit 112 to generate a display image for displaying the image of the occupant extracted in step S902 in the display area 1002 of the second captured image in the first captured image 1001 as shown in FIG. 10.

[0060] By the process of FIG. 9, the display control unit 110 can display a display image similar to the display image 401 shown in FIG. 4 on the display device 13.

[0061] [Third Embodiment] In the third embodiment, an example of the process when the display control device 100 detects the line of sight of the driver driving the vehicle 10 and changes the display method of the second captured image according to the driver's line of sight will be described.

[0062] FIG. 11 is a diagram showing a configuration example of a display control system according to the third embodiment. As shown in FIG. 11, the display control system 1 according to the third embodiment has a line of sight detection device 1101 and a line of sight detection unit 1102 in addition to the configuration of the display control system 1 according to the embodiment described with reference to FIG. 1.

[0063] The line of sight detection device 1101 is a device that detects the line of sight of a predetermined user. For example, the line of sight detection device 1101 extracts the pupils from the user's face image by a known image analysis technique, and detects the user's line of sight, viewpoint, etc. from its movement, direction, etc.

[0064] The line-of-sight detection unit 1102 is realized by, for example, a program executed by the CPU 501. Using the line-of-sight detection device 1101, it detects that the driver driving the vehicle 10 is gazing at the first captured image displayed on the display device 13. For example, when the driver's line of sight remains on the first captured image for a period exceeding a predetermined time, the line-of-sight detection unit 1102 determines that the driver is gazing at the first captured image.

[0065] Also, when the driver driving the vehicle 10 is gazing at the first captured image displayed on the display device 13, the display control unit 110 according to the third embodiment reduces or makes transparent the display area of the second captured image. Note that the functional configuration of the display control system 1 other than the above may be the same as the functional configuration of the display control system 1 according to the embodiment described in FIG. 1.

[0066] <Flow of processing> FIG. 12 is a flowchart showing an example of the processing of the display control device according to the third embodiment. Among the processing shown in FIG. 12, the processing of steps S601 to S603, S606, and S607 is the same as the processing of the first embodiment described in FIG. 6, so the description thereof is omitted here. Also, detailed descriptions of the same processing contents as those of the first and second embodiments are omitted here.

[0067] In step S1201, the display control unit 110 uses the display mode correction unit 111 to change the color scheme of the background of the passengers in the second captured image, and performs a transparency process on the background of the passengers with the changed color scheme. Note that this processing corresponds to the processing of steps S604 and S801 in FIG. 8.

[0068] In steps S1202 and S1203, the line-of-sight detection unit 1102 uses the line-of-sight detection device 1101 to detect the line of sight of the driver driving the vehicle 10, and determines whether the driver is gazing at the first captured image displayed on the display device 13. For example, as shown in FIG. 13(A), when the driver's line of sight (viewpoint) 1302 remains on the first captured image 1301 for a period exceeding a predetermined time, the line-of-sight detection unit 1102 determines that the driver is gazing at the first captured image.

[0069] As a specific example, the line-of-sight detection unit 1102 stores the detection results of the driver's line of sight 1302 by the line-of-sight detection device 1101 in a time series in the memory 502 or the like. Further, the line-of-sight detection unit 1102 detects that the driver is gazing at the first captured image from the driver's line of sight 1302 detected in step S1202 and the driver's line of sight 1302 stored in the memory 502 or the like in a time series.

[0070] When the driver is gazing at the first captured image, the display control unit 110 causes the process to proceed to step S1204. On the other hand, when the driver is not gazing at the first captured image, the display control unit 110 causes the process to proceed to step S1205.

[0071] When the process proceeds to step S1204, the display control unit 110 uses the display image generation unit 112 to reduce or make transparent the second display area. For example, as shown in FIG. 13(B), the display control unit 110 reduces the display area 1303 of the second captured image displayed in the first captured image 1301. Alternatively, as shown in FIG. 10, the display control unit 110 may make the display area 1002 of the second captured image transparent so that the first captured image can be visually recognized while maintaining the display area 1002 of the second captured image displayed in the first captured image 1001.

[0072] On the other hand, when the process proceeds to step S1205, the display control unit 110 sets the second captured image to a default display method without reducing or making it transparent.

[0073] In step S1206, the display control unit 110 generates a display image for displaying the first captured image 1301 and the display area 1303 of the second captured image by using the display image generation unit 112. As shown in FIG. 13(B), when reducing the display area 1303 of the second captured image, a second captured image with the color scheme of the passenger's background changed and subjected to a transparency process is displayed in the display area 1303 of the second captured image. On the other hand, as shown in FIG. 10, when making the display area 1002 of the second captured image transparent, it is not necessary to display the second captured image in the display area 1303 of the second captured image (or the transparency of the second captured image may be further increased for display).

[0074] Through the above processing, the display control device 100 can further reduce the area in the display area of the first captured image where the first captured image cannot be visually recognized due to the display area of the second captured image.

[0075] [Fourth Embodiment] In the fourth embodiment, an example will be described in which the display control device 100 has a state detection unit that detects the state of the passenger in the vehicle 10, and corrects the display mode of the background of the passenger in the second captured image displayed on the display device 13 according to the state of the passenger.

[0076] FIG. 14 is a diagram showing a configuration example of a display control system according to the fourth embodiment. As shown in FIG. 14, the display control system 1 according to the fourth embodiment has a state detection unit 1401 in addition to the configuration of the display control system 1 according to the embodiment described with reference to FIG. 1.

[0077] The state detection unit 1401 is realized by, for example, a program executed by the CPU 501, and detects the state of the occupants of the vehicle 10. As an example, the state detection unit 1401 analyzes the second captured image generated by the second imaging unit 130 and capturing the occupants in the rear seats of the vehicle 10, and detects the state of the occupants in the rear seats (for example, good condition, car sickness, fever, sleep, etc.). As another example, the state detection unit 1401 may detect the state of the occupants using various sensors such as a temperature sensor, a pulse sensor, and a motion sensor provided in the seats of the vehicle 10. As another example, the state detection unit 1401 may estimate the state of the occupants by inputting the second captured image generated by the second imaging unit 130 into a state estimation model that has been learned by machine learning.

[0078] Also, the display mode correction unit 111 according to the third embodiment corrects the display mode of the background of the occupants in the second captured image according to the state of the occupants detected by the state detection unit 1401. Preferably, the state detection unit 1401 detects the states of a plurality of occupants in the rear seats, and the display mode correction unit 111 corrects the display mode of the background for each occupant according to the states of the plurality of occupants. Note that the functional configuration of the display control system 1 other than the above may be the same as the functional configuration of the display control system 1 according to the embodiment described in FIG. 1.

[0079] <Flow of processing> FIG. 15 is a flowchart showing an example of the processing of the display control device according to the fourth embodiment. Note that, among the processing shown in FIG. 15, the processing of steps S601 to S603, S606, and S607 is the same as the processing of the first embodiment described in FIG. 6, and thus the description thereof is omitted here. Also, here, a detailed description of the processing content similar to that of the first to third embodiments is omitted.

[0080] In step S1501, the state detection unit 1401 detects the state of the occupants of the vehicle 10. For example, the state detection unit 1401 analyzes the second captured image generated by the second imaging unit 130 and detects the states of a plurality of occupants sitting in the rear seats of the vehicle 10.

[0081] In step S1502, the display control unit 110 uses the display mode correction unit 111 to change the color tone of the background of the plurality of passengers 1601 to 1603 in the second captured image 1600 according to the state of the passengers detected by the state detection unit 1401. For example, as shown in FIG. 16. In the example of FIG. 16, the display mode correction unit 111 sets the background color scheme of the passenger 1601 whose state is "good" to "green". Similarly, the display mode correction unit 111 sets the background color scheme of the passenger 1602 whose state is "fever" to "yellow", and the background of the passenger 1603 whose state is "carsickness" to "orange". Note that the "color" representing the state of the passenger or the "state" of the passenger to be detected may be set by the user (such as the driver) from a setting screen or the like.

[0082] In step S1503, the display control unit 110 uses the display mode correction unit 111 to perform a transparency process on the background of the passengers in the second captured image 1600 in which the color tone of the background of each passenger is changed.

[0083] In step S1504, the display control unit 110 uses the display image generation unit 112 to generate a display image for simultaneously displaying the first captured image and the second captured image.

[0084] By the process of FIG. 15, the display control unit 110 can display, for example, a display image 1604 as shown in FIG. 16 on the display device 13. Thereby, the user (such as the driver) can easily grasp the state of the passengers in the rear seat by viewing the second captured image 1600 displayed on the display device 13.

[0085] As described above, according to each embodiment of the present invention, in the display control device 100 that causes the display device 13 to display an image outside the vehicle and an image inside the vehicle, the user such as the driver can easily distinguish between the image outside the vehicle and the image inside the vehicle.

[0086] Note that the present invention is not limited to the above-described embodiments, and various modifications or applications are possible within the scope of the gist of the present invention described in the claims.

Description of Reference Numerals

[0087] 1 represents the control system 13 represents the display device (display unit) 110 represents the display control unit 120 represents the first imaging unit 402, 702 represent the first captured image 403, 703 represent the second captured image 1102 represents the gaze detection unit 1401 represents the state detection unit

Claims

A first imaging unit that generates a first captured image of the rear of the vehicle, A second imaging unit that generates a second captured image of the occupant in the rear seat, A display control unit that displays the first captured image and the second captured image on a display unit, and having, The display control unit, When simultaneously displaying the first captured image and the second captured image, the second captured image is displayed within the range of the first captured image displayed on the display unit, changing the color scheme of the background of the occupant in the second captured image, and making the background of the occupant with the changed color scheme transparent so that the first captured image can be visually recognized, A display control device.

2. Having a state detection unit that detects the state of the occupant, The display control unit changes the color scheme of the background of the occupant according to the state of the occupant. The display control device according to claim 1.

3. The state detection unit detects the states of a plurality of occupants in the rear seat, The display control unit changes the color scheme of the background for each of the plurality of occupants according to the states of the plurality of occupants. The display control device according to claim 2.

4. When the occupant is not shown in the second captured image, the display control unit stops displaying the second captured image. The display control device according to any one of claims 1 to 3.

5. Having a gaze detection unit that detects the gaze of the driver of the vehicle, The display control unit changes the display method of the second captured image according to the gaze of the driver. The display control device according to any one of claims 1 to 4.

6. When the driver is gazing at the first captured image, the display control unit reduces or makes transparent the display area of the second captured image. The display control device according to claim 5.

7. When the driver is not gazing at the first captured image, the display control unit displays the second captured image on the display unit in a default display method. The display control device according to claim 5 or 6.

8. A display control device mounted on a vehicle, A first imaging process that captures the rear of the vehicle and generates a first captured image, A second imaging process that captures the occupant in the rear seat and generates a second captured image, A display control process that displays the first captured image and the second captured image on a display unit, and executes, The display control process, When simultaneously displaying the first captured image and the second captured image, the second captured image is displayed within the range of the first captured image displayed on the display unit, Changing the color scheme of the background of the occupant in the second captured image and making the background of the occupant with the changed color scheme transparent so that the first captured image can be viewed. Display control method.

Images