Vehicle and vehicle control device

Abstract

This vehicle comprises: a first wheel; a second wheel; a vehicle body which is coupled to the first wheel and the second wheel and which can move by means of the first wheel and the second wheel; and a first imaging unit for capturing images outside the vehicle. An object to be monitored is set on the basis of captured images, and if monitoring start conditions are satisfied, monitoring processing is started to determine, on the basis of images captured by the first imaging unit, whether the object to be monitored is within a monitoring range, and if monitoring end conditions are satisfied, the monitoring processing is ended. The monitoring start conditions or the monitoring end conditions include a condition related to the operation of the vehicle.

Description

[Technical field]

[0001] The present disclosure relates to a vehicle and a vehicle control device. [Background technology]

[0002] Conventionally, a technique has been disclosed in which search conditions for search targets within a facility are set, and a person who is highly likely to be a search target is identified based on the search conditions, and the person is monitored (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-201758 A Summary of the Invention

[0004] The present disclosure has been made in consideration of the above, and has an object to appropriately control the start or end of a monitoring process in a vehicle.

[0005] A vehicle according to the present disclosure includes a first wheel, a second wheel, a vehicle body coupled to the first wheel and the second wheel and movable by the first wheel and the second wheel, and a first imaging unit that images the outside of the vehicle. a second imaging unit that captures an image of the interior of the vehicle; A vehicle equipped with Based on the image captured by the second imaging unit and the past image captured by the second imaging unit, A monitoring target is set, and when a monitoring start condition is met, a monitoring process is started to determine whether or not the monitoring target belongs to a monitoring range based on an image captured by the first imaging unit, and when a monitoring end condition is met, Monitoring Process The monitoring start condition or the monitoring end condition includes a condition related to the operation of the vehicle. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a plan view that shows a schematic diagram of a vehicle according to the present embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a hardware configuration of the vehicle control device according to the present embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a functional configuration of the vehicle control device. [Figure 4] FIG. 4 is a diagram illustrating an example of a monitoring range. [Diagram 5] FIG. 5 is a flowchart showing a procedure of a monitoring range setting process. [Figure 6] FIG. 6 is a flowchart showing a procedure of a manual monitoring target setting process. [Figure 7] FIG. 7 is a flowchart showing a monitoring target setting process by learning. [Figure 8] FIG. 8 is a flowchart showing a control procedure of the monitoring process. [Figure 9] FIG. 9 is a diagram showing a schematic configuration of a vehicle according to the second embodiment. [Figure 10] FIG. 10 is a diagram showing a configuration of a lighting unit in a vehicle interior in the second embodiment. [Figure 11] FIG. 11 is a diagram showing the configuration of an imaging section and an illumination section in the second embodiment. [Figure 12] FIG. 12 is a diagram illustrating a hardware configuration of a vehicle control device in the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating a part of the functional configuration of a vehicle control device in the second embodiment. [Figure 14] FIG. 14 is a diagram showing the position of the image acquisition area in the entire image in the second embodiment. [Figure 15] FIG. 15 is a diagram showing spatial changes of a plurality of light-emitting units in response to images at a plurality of timings in the second embodiment. [Figure 16] FIG. 16 is a diagram showing changes over time of a plurality of light-emitting units corresponding to images at a plurality of timings in the second embodiment. [Figure 17] FIG. 17 is a diagram showing a plurality of divided regions within an image acquisition area in the second embodiment. [Figure 18] FIG. 18 is a diagram showing a process for determining the luminance of a light-emitting unit in the second embodiment. [Figure 19] FIG. 19 is a flowchart showing the operation of the vehicle according to the second embodiment. [Figure 20] FIG. 20 is a flowchart showing the operation of the vehicle according to the second embodiment. [Figure 21] FIG. 21 is a diagram showing control of a plurality of light-emitting units according to an image and map information of an image acquisition area in a first modified example of the second embodiment. [Figure 22] FIG. 22 is a flowchart showing the operation of a vehicle according to a first modified example of the second embodiment. [Figure 23] FIG. 23 is a flowchart showing the operation of a vehicle according to the second modification of the second embodiment. [Figure 24] FIG. 24 is a schematic diagram illustrating an example of a vehicle including a communication control device according to the third embodiment. As illustrated in FIG. [Diagram 25] FIG. 25 is a diagram illustrating an example of a hardware configuration of the communication control device according to the third embodiment. As illustrated in FIG. [Figure 26] FIG. 26 is a diagram illustrating an example of a functional configuration of the communication control device according to the third embodiment. As illustrated in FIG. [Figure 27] FIG. 27 is a diagram showing icons, a seating chart, and seat data stored in the storage memory of the communication control device according to the third embodiment. [Figure 28] FIG. 28 is a flowchart of an example of a procedure of a transmission process by the communication control device according to the third embodiment. [Figure 29] FIG. 29 is a diagram illustrating an example of a display on a display device of a transmission process by the communication control device according to the third embodiment. [Diagram 30] FIG. 30 is a diagram illustrating an example of a display on a display device of a transmission process by the communication control device according to the third embodiment. [Diagram 31] FIG. 31 is a diagram illustrating an example of a display on a display device of a transmission process by the communication control device according to the third embodiment. [Diagram 32]FIG. 32 is a flowchart of an example of a procedure of a receiving process by the communication control device according to the third embodiment. [Diagram 33] FIG. 33 is a diagram illustrating an example of a display on a display device of a receiving process by the communication control device according to the third embodiment. [Diagram 34] FIG. 34 is a flowchart of an example of a procedure of a reply process by the communication control device according to the third embodiment. [Diagram 35] FIG. 35 is a diagram illustrating an example of a display on a display device of a reply process by the communication control device according to the third embodiment. [Diagram 36] FIG. 36 is a diagram illustrating an example of a display on a display device of a reply process by the communication control device according to the third embodiment. [Figure 37] FIG. 37 is a diagram illustrating an example of a display of a light-emitting device in a light-emitting process by the communication control device according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, embodiments of a vehicle and a vehicle control device according to the present disclosure will be described with reference to the drawings.

[0008] (First embodiment) 1 is a plan view showing a schematic diagram of a vehicle 1 according to the present embodiment. The vehicle 1 includes a vehicle body 2 and two pairs of wheels 3 (a pair of front tires 3f and a pair of rear tires 3r) arranged at the front and rear of the vehicle body 2.

[0009] The vehicle 1 can run using two pairs of wheels 3 arranged along a predetermined direction. In this case, the predetermined direction in which the two pairs of wheels 3 are arranged is the direction of movement of the vehicle 1, and the vehicle can move forward or backward by switching gears, etc.

[0010] The vehicle 1 also includes a vehicle drive unit 5, an exterior imaging unit 6, and an exterior speaker 30. The exterior imaging unit 6 is an example of a first imaging unit.

[0011] The vehicle drive unit 5 is a drive device mounted on the vehicle 1. The vehicle drive unit 5 is, for example, an engine, a motor, a drive unit for the wheels 3, or the like.

[0012] The outside imaging unit 6 is a camera that captures images of the outside of the vehicle. The outside imaging unit 6A is provided in the front part of the vehicle 1. The outside imaging unit 6A captures images of the area in front of the vehicle 1 and generates captured images. The outside imaging unit 6B is provided in the rear part of the vehicle 1. The outside imaging unit 6B captures images of the area behind the vehicle 1 and generates captured images. The outside imaging unit 6C and the outside imaging unit 6D are provided, for example, around the door mirrors of the vehicle 1. The outside imaging unit 6C and the outside imaging unit 6D capture images of the areas to the sides of the vehicle 1 and generate captured images.

[0013] In this embodiment, a form will be described as an example in which a plurality of outside-vehicle imaging units 6 (outside-vehicle imaging units 6A to 6D) are provided on the vehicle body 2 of the vehicle 1. Note that these outside-vehicle imaging units 6 are disposed at positions where they can capture images of the outside of the vehicle body 2.

[0014] The exterior speaker 30 is connected to the vehicle control device 20, and outputs sound in response to an audio signal output from the vehicle control device 20. The exterior speaker 30 is provided outside the passenger compartment of the vehicle 1, for example, in an engine room. The exterior speaker 30 outputs sound to the outside of the vehicle 1.

[0015] A seat 4 is provided in the cabin of the vehicle 1, in which a driver, who is a user of the vehicle 1, sits. That is, the seat 4 is a driver's seat. The driver can steer the vehicle 1 while seated in the seat 4 by operating a steering wheel 11 and an accelerator pedal and a brake pedal (not shown).

[0016] Inside the cabin of the vehicle 1, there are provided an interior imaging unit 7, an engine switch 10, a shift lever 12, a parking brake 13, a door lock device 14, an in-vehicle display device 15, a human detection sensor 16, and the like.

[0017] The interior imaging unit 7 is an example of a second imaging unit. The interior imaging unit 7 is a camera that captures images of the inside of a vehicle. The interior imaging unit 7 is provided inside the vehicle cabin. The interior imaging unit 7 captures, for example, an image of the surroundings of a rear seat 8 where a passenger sits. The interior imaging unit 7 captures an image of the surroundings of the rear seat 8 of the vehicle 1 and generates a captured image.

[0018] The engine switch 10 is a switch that the driver operates when starting the engine of the vehicle 1. The engine switch 10 is sometimes called an ignition switch. The driver operates the engine switch 10 to start or stop the engine of the vehicle 1. The operation of the engine switch 10 also controls the power supply to electronic devices mounted on the vehicle 1.

[0019] The shift lever 12 is a lever that is operated by the driver when changing the shift position. The movable range of the shift lever 12 includes, for example, a parking position, a reverse position, a neutral position, and a drive position.

[0020] When the shift lever 12 is in the parking position, the power of the engine of the vehicle 1 is not transmitted to the wheels 3, that is, the vehicle is in a parked state. When the shift lever 12 is in the reverse position, the vehicle 1 is in a state in which it can move backward. When the shift lever 12 is in the drive position, the vehicle 1 is in a state in which it can move forward.

[0021] The parking brake 13 is one of the brake mechanisms of the vehicle 1. The parking brake 13 is a manual control mechanism that allows the driver to manually stop the movement of the vehicle 1. When the parking brake 13 is applied, the movement of the vehicle 1 is stopped and the vehicle 1 is in a parked state. When the parking brake 13 is released, the vehicle 1 is in a movable state.

[0022] The door lock device 14 is a device that switches the state of the doors of the vehicle 1 between a locked state and an unlocked state. For example, the door of the vehicle 1 can be switched between a locked state and an unlocked state by a vehicle key or the like from outside the vehicle 1.

[0023] The in-vehicle display device 15 is an example of a display device. The in-vehicle display device 15 has not only an image display function, but also an audio output function, an input function, and the like. The image display function is, for example, an LCD (Liquid Crystal Display) or an OELD (Organic Electro-Luminescent Display). The audio output function is, for example, an in-vehicle speaker. The in-vehicle display device 15 is configured as a touch panel having a function of accepting operation input by a user. The in-vehicle display device 15 may also be configured to further include a switch or the like that accepts operation input by a user.

[0024] The in-vehicle display device 15 may also include a navigation device having a location information acquisition function and a route search function using map information.

[0025] The human detection sensor 16 is a sensor that detects the presence or absence of a human inside the vehicle body 2. The human detection sensor 16 is, for example, an infrared sensor, an image analysis system for captured images, a seating sensor based on weight, or the like.

[0026] The vehicle 1 is equipped with a communication device, which is not shown in FIG. 1, a device for detecting the open / closed state of the doors of the vehicle 1, and a device for detecting the state of the ignition.

[0027] In this embodiment, the vehicle 1 is equipped with a vehicle control device 20. The vehicle control device 20 is a device that can be installed in the vehicle 1. The vehicle control device 20 sets a monitoring range and a monitoring target, and determines whether or not the monitoring target has deviated from the monitoring range. Here, the monitoring target is, for example, a child riding in the vehicle 1. The vehicle control device 20 is, for example, an ECU (Electronic Control Unit) or an OBU (On Board Unit) provided inside the vehicle 1. Alternatively, the vehicle control device 20 may be an external device installed near the dashboard of the vehicle 1.

[0028] Next, a hardware configuration of the vehicle control device 20 will be described. Fig. 2 is a diagram showing an example of a hardware configuration of the vehicle control device 20 according to this embodiment. As shown in Fig. 2, the vehicle control device 20 includes an interface (I / F) 21, a CPU 22, a RAM 23, a ROM 24, and a HDD (Hard Disk Drive) 25. The interface 21, the CPU 22, the RAM 23, the ROM 24, and the HDD 25 are connected via a bus 26.

[0029] The interface 21 is one of various interfaces such as a communication interface, an image interface, an audio interface, a display interface, and an input interface.

[0030] The HDD 25 stores various information, for example, images captured by the outside-vehicle image capturing unit 6 and the inside-vehicle image capturing unit 7.

[0031] The CPU 22 executes various processes by executing programs stored in the ROM 24 or the HDD 25. The CPU 22 is an example of a processor.

[0032] Fig. 3 is a diagram showing an example of the functional configuration of the vehicle control device 20. The program executed by the vehicle control device 20 has a modular configuration including a monitoring range setting unit 201, a monitoring target setting unit 202, an information acquisition unit 203, and a monitoring control unit 204 shown in Fig. 3, and as actual hardware, the CPU 22 reads out and executes the program from a storage device such as the ROM 24 or the HDD 25, whereby the above-mentioned units are loaded onto a main storage device such as the RAM 23, and the monitoring range setting unit 201, the monitoring target setting unit 202, the information acquisition unit 203, and the monitoring control unit 204 are generated on the main storage device.

[0033] In addition, the vehicle control device 20 is connected to an exterior imaging unit 6, an interior imaging unit 7, a parking brake 13, a door lock device 14, an in-vehicle display device 15, an exterior speaker 30, a communication device 40, an ignition detection device 50 and a door opening / closing detection device 60.

[0034] The communication device 40 is a device configured with a communication device for connecting to a communication network such as the Internet. The communication device 40 may be a wireless LAN compatible communication device, an LTE (Long Term Evolution) compatible communication device, or a wired communication device that performs wired communication. The vehicle control device 20 can transmit and receive information to and from a terminal device such as a smartphone via the communication device 40.

[0035] The ignition detection device 50 is a sensor device that is connected to the engine switch 10 and detects the state of the ignition. The ignition detection device 50 outputs information indicating the state of the ignition switch.

[0036] The door opening / closing detection device 60 is a device that detects the opening / closing of the driver's seat side and passenger seat side doors of the front and rear rows of the vehicle 1. For example, the door opening / closing detection device 60 is a sensor that detects the opening / closing of a door. The door opening / closing detection device 60 is disposed, for example, at the hinge portion of each door. The door opening / closing detection device 60 detects that the opening angle has reached a level that allows passengers to get in and out of the vehicle. When it is detected that the opening angle of the door is sufficient for passengers to get in and out of the vehicle, it can be considered that a vehicle operation has been performed that allows the user of the vehicle 1 to get out of the vehicle. The user of the vehicle 1 is, for example, a passenger in the back seat.

[0037] The monitoring range setting unit 201 sets a monitoring range. This monitoring range is a range that takes into consideration the possibility that the monitoring target may get lost, and is a range that determines whether or not to perform notification processing to the monitoring target itself or a passenger (e.g., the driver) of the vehicle 1, and is a so-called geographical fence, a geofence. That is, if the monitoring target falls within the monitoring range, the vehicle control device 20 does not perform notification processing, but if the monitoring target deviates from the monitoring range, it performs notification processing.

[0038] For example, when a mode indicating setting a monitoring range is selected by a user's input operation on the in-vehicle display device 15, the monitoring range setting unit 201 starts the monitoring range setting process. In the monitoring range setting process, the monitoring range setting unit 201 receives, by a user's input operation on the in-vehicle display device 15, the selection of the outside-vehicle imaging unit 6 to be used during the monitoring process and input of information that defines the monitoring range of the selected outside-vehicle imaging unit 6. The information that defines the monitoring range of the selected outside-vehicle imaging unit 6 is, for example, distance information from the selected outside-vehicle imaging unit 6.

[0039] The monitoring range setting unit 201 sets the monitoring range based on the selection of the vehicle outside imaging unit 6 to be used during the monitoring process and information that defines the monitoring range of the selected vehicle outside imaging unit 6. An example of the monitoring range will now be described with reference to FIG. 4. FIG. 4 is a diagram illustrating an example of the monitoring range. As shown in FIG. 4, when it is input that the vehicle outside imaging units 6A to 6D will be used during the monitoring process and information indicating monitoring ranges AR1 to AR4, which are the monitoring ranges of each vehicle outside imaging unit 6, the monitoring range setting unit 201 sets the monitoring ranges AR1 to AR4 as the monitoring ranges.

[0040] The monitoring range setting unit 201 may only accept a selection of the vehicle outside imaging unit 6 to be used during the monitoring process, and may set the monitoring range of each vehicle outside imaging unit 6 to the imaging range that can be captured by each vehicle outside imaging unit 6. In this manner, the monitoring range setting unit 201 may determine the monitoring range based on the imaging range that can be captured by the vehicle outside imaging unit 6.

[0041] Next, the procedure for setting the monitoring range will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the procedure for setting the monitoring range.

[0042] The monitoring range setting unit 201 accepts a selection of the vehicle outside image capturing unit 6 to be used during monitoring processing (step S1). Then, the monitoring range setting unit 201 accepts an input of information that defines the monitoring range of each vehicle outside image capturing unit 6 (step S2). Then, the monitoring range setting unit 201 sets the monitoring range (step S3).

[0043] Returning to Fig. 3, the monitoring target setting unit 202 sets a monitoring target based on a captured image. There are two methods for setting a monitoring target: manual monitoring target setting, which is monitoring target setting without learning, and monitoring target setting by learning. First, manual monitoring target setting will be described.

[0044] When a mode indicating manual registration is selected by a user's input operation on the in-vehicle display device 15 and the vehicle exterior imaging unit 6 or vehicle interior imaging unit 7 that will capture images is selected, the monitoring target setting unit 202 starts the selected vehicle exterior imaging unit 6 or vehicle interior imaging unit 7 and causes them to perform imaging processing. The monitoring target setting unit 202 acquires captured images from the selected vehicle exterior imaging unit 6 or vehicle interior imaging unit 7, performs image analysis on the captured images, and sets the whole-body image and face image of a person included in the captured images as monitoring targets. The monitoring target setting unit 202 may also perform image analysis on captured images transmitted from a terminal device carried by the user, and set the whole-body image and face image of a person included in the captured images as monitoring targets.

[0045] Next, a procedure for manually setting a monitoring target will be described with reference to Fig. 6. Fig. 6 is a flowchart showing a procedure for manually setting a monitoring target.

[0046] The monitoring target setting unit 202 accepts a selection of the outside-vehicle image capturing unit 6 or the inside-vehicle image capturing unit 7 to be used for setting the monitoring target (step S11). Then, the monitoring target setting unit 202 activates the selected image capturing unit (step S12). The monitoring target setting unit 202 acquires an image captured from the activated image capturing unit (step S13). The monitoring target setting unit 202 performs image analysis on the captured image, and sets a whole-body image and a face image of a person included in the captured image as a monitoring target (step S14).

[0047] Next, the monitoring target setting by learning will be described. At a predetermined timing, such as immediately after the vehicle 1 starts traveling, the monitoring target setting unit 202 starts the vehicle interior imaging unit 7 and obtains a captured image of, for example, the surroundings of the back seat 8 from the vehicle interior imaging unit 7.

[0048] The monitoring target setting unit 202 performs image analysis on the captured image, extracts a whole-body image and a face image of a person included in the captured image, and stores the captured date and time, the whole-body image, and the face image as a monitoring target candidate in association with each other. As a method for extracting a whole-body image and a face image of a person included in a captured image, the technology described in JP 2020-178167 A can be applied.

[0049] The surveillance target setting unit 202 refers to the stored past surveillance target candidates and judges whether there are a threshold or more of facial images of the same face as the face indicated by the facial image, which were captured on different days. An example of the threshold is 10. The surveillance target setting unit 202 can apply the person matching technology described in the above-mentioned Patent Document 1 as a method for referring to the stored past surveillance target candidates and judging whether the facial images are the same face as the face indicated by the facial image.

[0050] If there are more than the threshold value, the surveillance target setting unit 202 sets the most recently stored face image and whole-body image of the surveillance target candidate as the surveillance target.

[0051] Next, a procedure for setting a monitoring target by learning will be described with reference to Fig. 7. Fig. 7 is a flowchart showing a procedure for setting a monitoring target by learning.

[0052] The monitoring target setting unit 202 acquires a captured image from the in-vehicle image capturing unit 7 (step S21). Next, if the monitoring target setting unit 202 cannot extract a person as a result of image analysis of the captured image (step S22: No), the processing ends. On the other hand, if the monitoring target setting unit 202 extracts a person as a result of image analysis of the captured image (step S22: Yes), the monitoring target setting unit 202 stores the image capture date and time, a whole-body image, and a face image as a monitoring target candidate in association with each other (step S23).

[0053] The surveillance target setting unit 202 refers to the stored past surveillance target candidates and determines whether there are 10 or more facial images of the same face as the face indicated by the facial image, each captured on a different day; if there are not 10 or more facial images (step S24: No), the process terminates.

[0054] If there are 10 or more facial images of the same face as the face indicated by the facial image, each captured on a different day (step S24: Yes), the surveillance target setting unit 202 sets the most recently stored facial image and full-body image of the surveillance target candidate as the surveillance target (step S25).

[0055] 3, the information acquisition unit 203 acquires vehicle information. The vehicle information is information that indicates the state of the vehicle. Specifically, the vehicle information includes parking brake information, door lock state information, door open / close information, ignition information, and the like.

[0056] The parking brake information is information indicating whether the parking brake 13 of the host vehicle is applied or released. The information acquisition unit 203 receives the side brake information from, for example, a sensor device that detects the state of the parking brake 13.

[0057] The door lock status information is information indicating whether the doors of the host vehicle are locked or unlocked. The door lock status information also includes information indicating that the doors have been locked from outside the vehicle 1. The information acquisition unit 203 receives the door lock status information from the door lock device 14 that detects the locked and unlocked states of the doors.

[0058] The door opening / closing information is information indicating whether the door is open or closed. The information acquiring unit 203 acquires the door opening / closing information from the door opening / closing detection device 60.

[0059] The ignition information is information that indicates the state of the ignition switch. The information acquisition unit 203 acquires the ignition information from the ignition detection device 50.

[0060] The monitoring control unit 204 executes a monitoring process to determine whether or not the monitoring target is within the monitoring range based on the image captured by the outside-vehicle image capture unit 6. The monitoring control unit 204 starts the monitoring process when a monitoring start condition is met, and ends the monitoring process when a monitoring end condition is met.

[0061] The monitoring control unit 204 sequentially acquires vehicle information from the information acquisition unit 203, and determines whether or not the monitoring start condition and the monitoring end condition are met based on the vehicle information. The monitoring control unit 204 may also start the vehicle exterior imaging unit 6 and the vehicle interior imaging unit 7, and further use the image analysis results of the captured images acquired from the vehicle exterior imaging unit 6 and the vehicle interior imaging unit 7 to determine whether or not the monitoring start condition and the monitoring end condition are met.

[0062] The monitoring start conditions are, for example, that the vehicle 1 is unlocked from the outside, the ignition switch is OFF, the doors are open, the parking brake is applied, and the monitoring target is outside the vehicle. Note that the monitoring start conditions may also include a monitoring start instruction given by a user's input operation on the in-vehicle display device 15 or a user's terminal.

[0063] The monitoring and control unit 204 refers to the door lock state information to determine whether or not the vehicle 1 has been locked from the outside. The monitoring and control unit 204 also refers to the ignition information to determine whether or not the ignition switch is OFF. The monitoring and control unit 204 also refers to the door opening and closing information to determine whether or not the door of the vehicle 1 has been opened. The monitoring and control unit 204 also refers to the parking brake information to determine whether or not the parking brake is applied.

[0064] The monitoring control unit 204 performs image analysis on the captured image acquired from the vehicle exterior image capturing unit 6, and detects that the monitoring target is outside the vehicle if the captured image contains a facial image identical to that of the monitoring target.

[0065] The monitoring control unit 204 starts the monitoring process when the above monitoring start condition is met. Note that the monitoring control unit 204 may start the monitoring process on the condition that the monitoring start condition is met and the monitoring end condition is not met.

[0066] The monitoring control unit 204 activates the vehicle exterior imaging unit 6, which has one of the monitoring ranges as its imaging range, acquires an image from the vehicle exterior imaging unit 6, and searches for a monitoring target from the acquired image. The monitoring control unit 204 may search for a monitoring target using not only the face part of the acquired image but also the whole-body image. The technology for searching for a monitoring target using the face part and the whole-body image may be the technology described in JP 2020-178167 A.

[0067] The monitoring control unit 204 searches for a monitoring target and judges whether or not the monitoring target belongs to a monitoring range. The monitoring control unit 204 may determine whether or not the monitoring target belongs to a monitoring range by specifying a distance from the vehicle 1 to the monitoring target using a captured image including the monitoring target. For example, the technology for specifying the distance using a captured image may be the technology described in Japanese Patent Application Laid-Open No. 2007-188417, which determines the size of the entire body of the monitoring target from a pattern of a part of the monitoring target.

[0068] The monitoring control unit 204 determines whether the monitoring target belongs to the monitoring range based on the distance from the vehicle 1 to the monitoring target, and if the monitoring target belongs to the monitoring range, it continues to determine again whether the monitoring target belongs to the monitoring range using the captured image obtained from the exterior imaging unit 6.

[0069] If the monitoring target is not found in any of the captured images acquired from the vehicle outside image capture unit 6 or if the monitoring target does not belong to the monitoring range, the monitoring control unit 204 performs a notification process.

[0070] As a notification process, the monitoring control unit 204 outputs a voice via the exterior speaker 30 instructing the monitored subject to return toward the vehicle 1. The voice may be a pre-recorded voice of the driver or the like. As a notification process, the monitoring control unit 204 may also transmit and output message information via the communication device 40 to a mobile terminal carried by the monitored subject, instructing the monitored subject to return toward the vehicle 1.

[0071] In addition, as a notification process, the monitoring control unit 204 may display and output on the in-vehicle display device 15 map information including information indicating the current location and an image captured when the monitoring target was most recently within the monitoring range.

[0072] In addition, if the monitoring target is not found multiple times in any of the captured images obtained from the exterior imaging unit 6, or if the monitoring target does not belong to the monitoring range for a certain period of time, the monitoring control unit 204 may, as a notification process, display on the in-vehicle display device 15 of the vehicle 1 that the monitoring target cannot be recognized or has deviated from the monitoring range.

[0073] By performing the notification process described above, when the monitoring target is no longer within the monitoring range, the vehicle 1 first calls out to the monitoring target to encourage it to return voluntarily, and if the monitoring target still does not return to the monitoring range, it outputs information to notify the driver or other passengers in the vehicle 1. This allows the vehicle 1 to prevent the monitoring target located around the vehicle 1, such as playing around the vehicle 1 while out and about, from moving away from the vehicle 1.

[0074] The monitoring control unit 204 ends the monitoring process when the monitoring end condition is met. The monitoring end condition is, for example, when the vehicle 1 is locked from the outside, when the parking brake is turned off, or when the monitoring target returns to the vehicle. The monitoring end condition may include a monitoring end instruction given by a user's input operation on the in-vehicle display device 15 or a user's terminal.

[0075] In addition, when the monitoring control unit 204 terminates the monitoring process when the vehicle 1 is locked from outside or the parking brake is turned off, the monitoring control unit 204 may cause the in-vehicle display device 15 or the user's terminal device to display and output information indicating that the monitoring process is being executed.

[0076] The monitoring and control unit 204 refers to the door lock state information to determine whether or not the doors are locked from the outside of the vehicle 1. The monitoring and control unit 204 also refers to the parking brake information to determine whether or not the parking brake is OFF.

[0077] The monitoring control unit 204 performs image analysis on the captured image acquired from the in-vehicle image capturing unit 7, and detects that the monitoring target has returned inside the vehicle if the captured image contains a facial image identical to that of the monitoring target.

[0078] Next, the control procedure of the monitoring process will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the control procedure of the monitoring process.

[0079] The monitoring control unit 204 determines whether or not the monitoring start condition is met (step S31), and ends the process if the monitoring start condition is not met (step S31: No). Here, the monitoring start condition is when the ignition is OFF, the parking brake 13 is ON, the door is open, and the monitoring target is found in any of the captured images acquired by the vehicle exterior image capture unit 6.

[0080] If the monitoring start condition is met (step S31: Yes), the monitoring control unit 204 proceeds to step S32. In step S32, the monitoring control unit 204 determines whether the monitoring end condition is not met (step S32). The monitoring end conditions are when the vehicle 1 is locked from the outside, when the parking brake is turned off, when a request to stop the monitoring process is made from the in-vehicle display device 15 by a user's input operation, or when the monitored object returns to the inside of the vehicle.

[0081] If the monitoring end condition is not met in step S32 (step S32: Yes), the monitoring control unit 204 performs a search process for the monitoring target from the captured images acquired from the outside-vehicle image capture unit 6 (step S33). If the monitoring control unit 204 can recognize the monitoring target as a result of the search process for the monitoring target (step S34: Yes), and if the monitoring target is within the monitoring range (step S35: Yes), the monitoring control unit 204 proceeds to step S32 without performing a notification process.

[0082] If the monitoring control unit 204 cannot recognize the monitoring target as a result of the search process for the monitoring target (step S34: No), or if the monitoring target is not within the monitoring range in step S35 (step S35: No), it performs notification processing (step S36) and proceeds to step S32. If the monitoring end condition is met in step S32 (step S32: No), the process ends.

[0083] In the vehicle control device 20 mounted on the vehicle 1 of this embodiment, the monitoring target setting unit 202 sets a monitoring target based on a captured image, and when a monitoring start condition is met, the monitoring control unit 204 starts a monitoring process to determine whether or not the monitoring target belongs to a monitoring range based on an image captured by the outside-vehicle imaging unit 6. In addition, when a monitoring end condition is met, the monitoring control unit 204 ends the monitoring process. The monitoring start condition or monitoring end condition includes conditions related to the operation of the vehicle, such as the state of the parking brake 13.

[0084] In this way, vehicle 1 controls the monitoring process in conjunction with the operation of vehicle 1, and therefore can control the start or end of the monitoring process without requiring any operation specific to the start or end of the monitoring process.

[0085] Furthermore, in the vehicle 1, the monitoring start conditions include the parking brake 13 being applied. When the vehicle 1 is parked, there is a possibility that an occupant such as a child may move away from the vehicle 1, and it is considered necessary to perform monitoring processing. Based on this, by including the parking brake 13, which is a condition based on the parked state, in the monitoring start conditions, the vehicle 1 can appropriately control the start of monitoring processing.

[0086] Furthermore, the monitoring start conditions for the vehicle 1 include a state in which the vehicle is not locked from the outside. If the vehicle is locked from the outside, it is presumed that the driver and other occupants will leave the vehicle 1 while the vehicle 1 is parked, and therefore the vehicle 1 does not need to perform monitoring processing. In this way, the vehicle 1 can avoid performing monitoring processing at a timing when it is not necessary.

[0087] Furthermore, in the vehicle 1, the monitoring start condition includes the opening of a door of the vehicle 1. In this way, by including in the monitoring start condition the opening of a door that is assumed to indicate that the monitoring target is leaving the vehicle 1, the vehicle 1 can execute the monitoring process at a more appropriate timing.

[0088] Furthermore, in the vehicle 1, the monitoring start conditions include extraction of a monitoring target based on an image captured by the vehicle exterior image capture unit 6. In this way, by including in the monitoring start conditions a condition indicating that the monitoring target is away from the vehicle 1, the vehicle 1 can execute the monitoring process at a more appropriate timing.

[0089] Furthermore, in the vehicle 1, the monitoring end conditions include the release of the parking brake 13. In this way, by including in the monitoring end conditions the conditions under which it is estimated that the vehicle 1 is no longer in a parked state and there is no need for monitoring processing, the vehicle 1 can appropriately control the monitoring processing.

[0090] Furthermore, in the vehicle 1, the monitoring end conditions include locking from outside the vehicle 1. When the vehicle 1 is locked from outside, it is presumed that the driver and other occupants will leave the vehicle 1, so that the vehicle 1 can be prevented from executing the monitoring process at a timing when the monitoring process is not required.

[0091] Furthermore, the vehicle 1 performs a notification process when it determines that the monitoring target does not belong to the monitoring range based on the image captured by the outside-vehicle image capturing unit 6. In this way, the vehicle 1 can perform a process to prevent the monitoring target from getting lost by notifying when the monitoring target goes outside the monitoring range.

[0092] Furthermore, in the vehicle 1, a monitoring target is set based on the image captured by the in-vehicle imaging unit 7 and the image captured in the past by the in-vehicle imaging unit 7. In this way, the vehicle 1 sets a monitoring target based on the image captured in the past, and therefore, by automatically setting a person who frequently rides in the vehicle as a monitoring target, it is possible to appropriately set a monitoring target without requiring complicated operations.

[0093] Furthermore, in the vehicle 1, the monitoring range is set based on the imaging range of the vehicle outside imaging unit 6. In this manner, the vehicle 1 can appropriately set the monitoring range without requiring complicated operations.

[0094] In the above embodiment, the monitoring start conditions are that the vehicle 1 is unlocked from the outside, the ignition switch is OFF, the doors are open, the parking brake is engaged, and the monitoring target is detected to be outside the vehicle, but this is not limited to the above. For example, additional conditions may be added, or only some of the monitoring start conditions in the above embodiment may be set as the monitoring start conditions.

[0095] In the above embodiment, the conditions are when the vehicle 1 is locked from the outside, when the parking brake is turned off, or when the monitoring target returns to the vehicle, but the conditions are not limited to this. Additional conditions may be added, or only some of the monitoring end conditions in the above embodiment may be set as the monitoring end conditions.

[0096] Also, the state of the shift lever 12 may be detected to determine whether the vehicle is in a parked state or not, and the result of this determination may be used as the monitoring start condition or monitoring end condition.

[0097] Also, instead of determining whether the door is locked from the outside, it may be possible to determine whether or not there are no occupants based on the detection result by the human detection sensor 16.

[0098] In the above embodiment, the case where the monitoring target is a child has been described, but the monitoring target may be an elderly person, a pet, etc. Also, in the above embodiment, the case where the vehicle 1 extracts a monitoring target candidate using an image captured by the in-vehicle imaging unit 7 around the rear seat 8, but the in-vehicle imaging unit 7 may capture an image of the passenger seat surroundings, and the monitoring target candidate may be extracted using the image captured around the passenger seat.

[0099] The program executed by the vehicle control device 20 of this embodiment is provided in an installable or executable format file recorded on a computer-readable recording medium such as an optical recording medium such as a DVD (Digital Versatile Disk), a USB memory, or a semiconductor memory device such as an SSD (Solid State Disk).

[0100] The program executed by the vehicle control device 20 of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The program executed by the vehicle control device 20 of the present embodiment may be provided or distributed via a network such as the Internet.

[0101] The program for the vehicle control device 20 of this embodiment may be provided by being pre-installed in a ROM or the like.

[0102] Second embodiment The vehicle according to the second embodiment has a vehicle body with a roof, and a vehicle cabin is formed inside the vehicle body. The vehicle cabin is a space in which passengers such as a driver and passengers should ride. The main function of the vehicle is to travel with passengers in the vehicle cabin and transport the passengers to their destination. When traveling in such a vehicle, passengers may request an increased sense of speed, immersion, and realism.

[0103] When a vehicle with passengers inside moves forward, the passengers inside the vehicle can see the scenery through the front and side windows, and feel as if the scenery ahead is approaching. For example, when driving on a road inside a tunnel, the tunnel lights approach from afar, and as the vehicle passes under the lights, the interior of the vehicle changes color from the front to the back, and then becomes dark again. Also, when driving through sunlight filtering through the trees, bright and dark areas are repeated, and the brightness inside the vehicle changes accordingly.

[0104] Since the vehicle runs on the ground, the passengers inside the vehicle will pass through the road scenery they see ahead, but they cannot see the scenery above because it is blocked by the roof. If an environment similar to the scenery could be provided above the passengers, it is expected that the sense of speed, immersion, and realism of the vehicle's running will be enhanced, and the entertainment value inside the vehicle will be improved.

[0105] Therefore, in the second embodiment, in a vehicle, multiple light-emitting elements are arranged in a row along the direction of travel on the ceiling of the passenger compartment, and the light-emitting state of each of the multiple light-emitting elements is controlled according to an image captured of the area in front of the vehicle, thereby improving the entertainment value within the passenger compartment.

[0106] Specifically, in a vehicle, the vehicle body has a roof portion covering the vehicle interior. The roof portion has an outer surface and an inner surface. An imaging unit is arranged at the front upper portion of the vehicle interior, and an illumination unit is provided on the inner surface of the roof portion, in which a plurality of light-emitting units are arranged in a row along the traveling direction. The imaging unit is capable of capturing an image of an image acquisition area. The image acquisition area is an area at a height corresponding to the roof portion in front of the vehicle body. The image acquisition area may be a part of the entire area included in the angle of view of the imaging unit. The illumination unit changes the light emission state of each of the plurality of light-emitting units according to an image of the image acquisition area. The imaging unit captures images of the image acquisition area for a plurality of timings. The plurality of timings may correspond to a plurality of points arranged at a predetermined distance interval along which the vehicle should travel. The illumination unit changes the light emission state of the plurality of light-emitting units over time according to a temporal change in the attribute of the image of the image acquisition area at the plurality of timings. When viewed from inside the vehicle interior, the illumination unit may change the light emission state of the plurality of light-emitting units so as to flow in a direction opposite to the traveling direction. Furthermore, the lighting unit changes the light emission state of the multiple light-emitting units spatially in response to a temporal change in the attributes of the image in the image acquisition area at multiple timings. The lighting unit may change the light emission state of the multiple light-emitting units in response to the attributes of the image in the image acquisition area. This makes it possible to realize a lighting environment on the inner surface of the roof that conforms to the scenery above, thereby enhancing the sense of speed, immersion, and realism when the vehicle is traveling. This makes it possible to improve the entertainment value inside the vehicle cabin.

[0107] More specifically, the vehicle is configured as shown in Figs. 9 to 11. Fig. 9 is a diagram showing a schematic configuration of a vehicle according to a second embodiment. Fig. 10 is a diagram showing a configuration of an illumination unit in the second embodiment. Fig. 11 is a diagram showing configurations of an imaging unit and an illumination unit in the second embodiment. In the following, the traveling direction of the vehicle 1 is defined as the X direction, the vehicle width direction is defined as the Y direction, and the direction perpendicular to the X direction and the Y direction is defined as the Z direction.

[0108] The vehicle 1 includes a plurality of wheels 42-1 to 42-4, a vehicle body 43, an imaging unit 44, a lighting unit 45, and a vehicle control device 100.

[0109] The wheels 42-1 to 42-4 are each rotatable around the Y axis. The wheels (second wheels) 42-3 and 42-4 are disposed on the -X side of the wheels (first wheels) 2-1 and 2-2, and accordingly, the +X side axle and the -X side axle (not shown) are disposed. The wheels 42-1 and 42-2 are respectively connected to the -Y side and +Y side ends of the +X side axle extending in the Y direction. The wheels 42-3 and 42-4 are respectively connected to the -Y side and +Y side ends of the -X side axle extending in the Y direction. In FIG. 9, a configuration in which the vehicle 1 has four wheels 42 is illustrated, but the number of wheels 42 may be three or less, or five or more.

[0110] The car body 43 rotatably supports an axle, and the wheels 42-1 to 42-4 are coupled to the axle. The car body 43 can move in the X direction by the rotation of the wheels 42-1, 42-2 and the wheels 42-3, 42-4. The car body 43 forms a vehicle interior 46. The car body 43 has a roof portion 43a and a plurality of windows 43b, 43c-1, 43c-2, 43d-1, 43d-2, 43e-1, 43e-2, 43f. The roof portion 43a covers the vehicle interior 46 from the +Z side, and defines the boundary of the vehicle interior 46 on the +Z side. The roof portion 43a has a surface on the +Z side as an outer surface, and a surface on the -Z side as an inner surface. The surface on the -Z side of the roof portion 43a forms a ceiling 46a of the vehicle interior 46. Windows 43b, 43c-1, 43c-2, 43d-1, 43d-2, 43e-1, 43e-2, and 43f define the boundaries of the +X side, -Y side, +Y side, -Y side, +Y side, -Y side, +Y side, -X side of vehicle interior 46, respectively.

[0111] An occupant in vehicle cabin 46 can see the scenery on the +X side, -Y side, +Y side, -Y side, +Y side, -Y side, +Y side, -X side through windows 43b, 43c-1, 43c-2, 43d-1, 43d-2, 43e-1, 43e-2, and 43f, but cannot see the scenery on the +Z side because it is blocked by roof portion 43a. The occupant is, for example, driver 300 or a passenger (not shown) in vehicle cabin 46.

[0112] The imaging unit 44 is disposed at a position between the +X side and the +Z side in the vehicle interior 46. The imaging unit 44 may be disposed adjacent to the -X side of the window 43b, or may be disposed near the end of the window 43b on the +Z side. The imaging unit 44 has a camera 44a. The imaging surface of the camera 44a faces the +X side, and its optical axis extends along the X direction as shown by the dashed line in FIG. 9. The optical axis of the camera 44a may be slightly inclined toward the +Z side with respect to the XY plane passing through the center of the camera 44a.

[0113] The imaging unit 44 can capture images of the outside of the vehicle body 43, and can capture images of the image acquisition areas IM1 and IM2. The imaging unit 44 may have one camera 44a capable of capturing images of both the image acquisition areas IM1 and IM2, or may have multiple cameras 44a capable of capturing images of the image acquisition areas IM1 and IM2, respectively. Fig. 11 illustrates a configuration in which the imaging unit 44 has one camera 44a capable of capturing images of both the image acquisition areas IM1 and IM2.

[0114] The imaging range of the imaging unit 44 includes a space away from the outer surface of the roof portion 43a, based on the ground 200 on which at least one of the wheels 42-1, 42-2 and the wheels 42-3, 42-4 is in contact with the ground. The image acquisition areas IM1, IM2 are areas at a height corresponding to the roof portion 43a in the front of the vehicle body 43, and may include a position whose height from the ground 200 is approximately equal to the height of the ceiling of the vehicle interior 46, as shown by a dotted line in FIG. 9. The image acquisition areas IM1, IM2 are areas located higher than a horizontal plane based on the eye position of the driver 300. A straight line connecting the eye position of the driver 300 and the center of the image acquisition areas IM1, IM2 extends along the X direction, as shown by a two-dot chain line in FIG. 9, and may be slightly inclined toward the +Z side with respect to the XY plane. The image acquisition areas IM1, IM2 may be a part of the entire area included in the angle of view of the camera 44a.

[0115] The illumination unit 45 is disposed at a position on the +Z side in the vehicle interior 46. The illumination unit 45 illuminates the vehicle interior 46. The illumination unit 45 has a light-emitting unit group 51, a light-emitting unit group 52, a light-emitting control circuit 53, a light-emitting control circuit 54 (see FIG. 12), a light source 55, and a light source 56.

[0116] The light-emitting unit group 51 is disposed in a region extending from the +X side to the -X side at a position on the +Z side in the vehicle interior 46. The light-emitting unit group 51 has a plurality of light-emitting units GR1 to GR3. In Figs. 9 to 11, a configuration in which the light-emitting unit group 51 has three light-emitting units GR1 to GR3 is illustrated as an example, but the number of light-emitting units in the light-emitting unit group 51 may be two, or four or more. The plurality of light-emitting units GR1 to GR3 of the light-emitting unit group 51 are disposed in a row along the X direction on the ceiling in the vehicle interior 46. The arrangement of the plurality of light-emitting units GR1 to GR3 of the light-emitting unit group 51 does not necessarily have to be a straight line, and may be a gentle curve in order to match the shape of the roof portion 43a of the vehicle body 43.

[0117] Each of the light-emitting units GR1 to GR3 of the light-emitting unit group 51 may have a plurality of light sources arranged in a row along the X direction. The light-emitting unit GR1 has a plurality of light sources D1 to D5 arranged in a row along the X direction. The light-emitting unit GR2 has a plurality of light sources D6 to D10 arranged in a row along the X direction. The light-emitting unit GR3 has a plurality of light sources D11 to D15 arranged in a row along the X direction.

[0118] The light emitting units GR1 to GR3 of the light emitting unit group 51 may be arranged slightly on the +Y side from the center in the Y direction on the ceiling of the vehicle interior 46. The light sources D1 to D15 of the light emitting unit group 51 are, for example, LEDs (Light Emission Diodes). The brightness of the light sources D1 to D15 of the light emitting unit group 51 can be adjusted by changing at least one of the duty ratio of the voltage and the magnitude of the current flowing through the light emitting unit group 51. The light emitting color of the light sources D1 to D15 of the light emitting unit group 51 may be colored or may be a single color such as R (red), G (green), B (blue), or white. In the following, the light emitting color of the light sources D1 to D15 of the light emitting unit group 51 is mainly illustrated as a colored light. The light emitting units GR1 to GR3 of the light emitting unit group 51 may be LED displays, liquid crystal displays, organic EL displays, etc. extending along the X direction instead of a plurality of light sources arranged in a row along the X direction.

[0119] The image acquisition area IM1 corresponds to the multiple light-emitting elements GR1 to GR3 of the light-emitting unit group 51. A straight line along the direction in which the multiple light-emitting elements GR1 to GR3 of the light-emitting unit group 51 are arranged passes through the image acquisition area IM1 as shown by dotted lines in Figs. 9 to 11. The straight line along the direction in which the multiple light-emitting elements GR1 to GR3 of the light-emitting unit group 51 are arranged may further pass near the vanishing point in the angle of view of the imaging unit 44. The illumination unit 45 can change the light-emitting state of each of the multiple light-emitting elements GR1 to GR3 of the light-emitting unit group 51 depending on the image of the image acquisition area IM1 captured by the imaging unit 44.

[0120] For example, the imaging unit 44 captures images of the image acquisition area IM1 at a plurality of timings. The plurality of timings may correspond to a plurality of points spaced at predetermined distances along which the vehicle 1 should travel.

[0121] Under the control of the vehicle control device 100, the light emission control circuit 53 changes the light emission state of each of the light emitting units GR1-GR3 of the light emitting unit group 51 in response to the movement of the vehicle body 43 along the traveling direction and the image captured by the imaging unit 44. The light emission control circuit 53 changes the light emission state of each of the light emitting units GR1-GR3 of the light emitting unit group 51 in response to the movement of the vehicle body 43 along the traveling direction and the image captured by the imaging unit 44 a predetermined time ago. This predetermined time may change according to the moving speed of the vehicle 1 in a predetermined direction.

[0122] The light emission control circuit 53 changes the light emission state of the light-emitting elements GR1 to GR3 of the light-emitting element group 51 over time in response to a change over time in the attribute of the image in the image acquisition area IM1. The light emission control circuit 53 changes the light emission state of the light-emitting elements GR1 to GR3 of the light-emitting element group 51 so that they flow in the opposite direction to the traveling direction when viewed from inside the vehicle interior 46 in response to a change over time in the attribute of the image in the image acquisition area IM1. The light emission control circuit 53 may change the light and dark pattern of the light-emitting elements GR1 to GR3 of the light-emitting element group 51 so that they flow in the opposite direction to the traveling direction when viewed from inside the vehicle interior 46 in response to a change over time in the attribute of the image in the image acquisition area IM1. The light emission control circuit 53 may change the color pattern of the light-emitting elements GR1 to GR3 of the light-emitting element group 51 so that they flow in the opposite direction to the traveling direction when viewed from inside the vehicle interior 46 in response to a change over time in the attribute of the image in the image acquisition area IM1.

[0123] Further, under the control of the vehicle control device 100, the light emission control circuit 53 spatially changes the light emission state of the multiple light-emitting elements GR1-GR3 of the light-emitting unit group 51 in response to a temporal change in the attribute of the image of the image acquisition area IM1. The light emission control circuit 53 changes the light emission state of the multiple light-emitting elements GR1-GR3 of the light-emitting unit group 51 in correspondence with the attribute of the image of the image acquisition area IM1. The light emission control circuit 53 may change the light and dark pattern of the multiple light-emitting elements GR1-GR3 of the light-emitting unit group 51 in correspondence with the luminance pattern of the image of the image acquisition area IM1. The light emission control circuit 53 may change the color pattern of the multiple light-emitting elements GR1-GR3 of the light-emitting unit group 51 in correspondence with the pattern of color component values ​​of the image of the image acquisition area IM1.

[0124] The light-emitting unit group 52 is disposed in a region extending from the +X side to the -X side at a position on the +Z side in the vehicle interior 46. The light-emitting unit group 52 has a plurality of light-emitting units GR1 to GR3. In Figs. 9 to 11, a configuration in which the light-emitting unit group 52 has three light sources D1 to D15 is illustrated, but the number of light sources that the light-emitting unit group 52 has may be two, or may be four or more. The light-emitting units GR1 to GR3 of the light-emitting unit group 52 are aligned in the Y direction with respect to the light-emitting units GR1 to GR3 of the light-emitting unit group 51. The light-emitting units GR1 to GR3 of the light-emitting unit group 52 are arranged in a row along the X direction on the ceiling in the vehicle interior 46. The arrangement of the light-emitting units GR1 to GR3 of the light-emitting unit group 52 does not necessarily have to be a straight line, and may be a gentle curve in order to match the shape of the roof portion 43a of the vehicle body 43.

[0125] Each of the light-emitting units GR1 to GR3 of the light-emitting unit group 52 may have a plurality of light sources arranged in a row along the X direction. The light-emitting unit GR1 has a plurality of light sources D1 to D5 arranged in a row along the X direction. The light-emitting unit GR2 has a plurality of light sources D6 to D10 arranged in a row along the X direction. The light-emitting unit GR3 has a plurality of light sources D11 to D15 arranged in a row along the X direction.

[0126] The light emitting units GR1 to GR3 of the light emitting unit group 52 may be arranged slightly on the -Y side from the center in the Y direction on the ceiling of the vehicle interior 46. The light sources D1 to D15 of the light emitting unit group 52 are, for example, LEDs (Light Emission Diodes). The brightness of the light sources D1 to D15 of the light emitting unit group 52 can be adjusted by changing at least one of the duty ratio of the voltage and the magnitude of the current flowing through the light emitting unit group 52. The light emitting color of the light sources D1 to D15 of the light emitting unit group 52 may be colored or may be a single color such as R (red), G (green), B (blue), or white. In the following, the case where the light emitting color of the light sources D1 to D15 of the light emitting unit group 52 is colored will be mainly illustrated. The light emitting units GR1 to GR3 of the light emitting unit group 52 may be LED displays, liquid crystal displays, organic EL displays, etc. extending along the X direction instead of a plurality of light sources arranged in a row along the X direction.

[0127] The image acquisition area IM2 corresponds to the multiple light-emitting elements GR1 to GR3 of the light-emitting unit group 52. A straight line along the direction in which the multiple light-emitting elements GR1 to GR3 of the light-emitting unit group 52 are arranged passes through the image acquisition area IM2 as shown by dotted lines in Figs. 9 to 11. The straight line along the direction in which the multiple light-emitting elements GR1 to GR3 of the light-emitting unit group 52 are arranged may further pass near the vanishing point in the angle of view of the imaging unit 44. The illumination unit 45 can change the light-emitting state of each of the multiple light-emitting elements GR1 to GR3 of the light-emitting unit group 52 depending on the image of the image acquisition area IM2 captured by the imaging unit 44.

[0128] For example, the imaging unit 44 captures images of the image acquisition area IM2 at a plurality of timings. The plurality of timings may correspond to a plurality of points spaced at predetermined distances along which the vehicle 1 should travel.

[0129] The light emission control circuit 54, under the control of the vehicle control device 100, changes the light emission state of each of the light emitting units GR1-GR3 of the light emitting unit group 51 in response to the movement of the vehicle body 43 along the traveling direction and the image captured by the imaging unit 44. The light emission control circuit 53 changes the light emission state of each of the light emitting units GR1-GR3 of the light emitting unit group 51 in response to the movement of the vehicle body 43 along the traveling direction and the image captured by the imaging unit 44 a predetermined time ago. This predetermined time may change according to the moving speed of the vehicle 1 in a predetermined direction.

[0130] The light emission control circuit 54 changes the light emission state of the light-emitting elements GR1 to GR3 of the light-emitting element group 52 over time in response to a change over time in the attribute of the image in the image acquisition area IM2. The light emission control circuit 54 changes the light emission state of the light-emitting elements GR1 to GR3 of the light-emitting element group 52 so that they flow in the opposite direction to the traveling direction when viewed from inside the vehicle interior 46 in response to a change over time in the attribute of the image in the image acquisition area IM2. The light emission control circuit 54 may change the light and dark pattern of the light-emitting elements GR1 to GR3 of the light-emitting element group 52 so that they flow in the opposite direction to the traveling direction when viewed from inside the vehicle interior 46 in response to a change over time in the attribute of the image in the image acquisition area IM2. The light emission control circuit 54 may change the color pattern of the light-emitting elements GR1 to GR3 of the light-emitting element group 52 so that they flow in the opposite direction to the traveling direction when viewed from inside the vehicle interior 46 in response to a change over time in the attribute of the image in the image acquisition area IM2.

[0131] Further, under the control of the vehicle control device 100, the light emission control circuit 54 spatially changes the light emission states of the multiple light-emitting elements GR1-GR3 of the light-emitting unit group 52 in response to temporal changes in the attributes of the image in the image acquisition area IM2. The light emission control circuit 54 changes the light emission states of the multiple light-emitting elements GR1-GR3 in correspondence with the attributes of the image in the image acquisition area IM2. The light emission control circuit 54 may change the light and dark pattern of the multiple light-emitting elements GR1-GR3 of the light-emitting unit group 52 in correspondence with the luminance pattern of the image in the image acquisition area IM2. The light emission control circuit 54 may change the color pattern of the multiple light-emitting elements GR1-GR3 of the light-emitting unit group 52 in correspondence with the pattern of color component values ​​of the image in the image acquisition area IM2.

[0132] 10, the light sources 55, 56 are disposed at positions on the +Z side and +X side in the vehicle interior 46. The light sources 55, 56 are capable of illuminating the vicinity of the driver 300's seat and the vicinity of the passenger seat, respectively.

[0133] The vehicle control device 100 is an information processing device that can be mounted on the vehicle 1, and is disposed at any position within the vehicle body 43, for example, at the position indicated by the dotted line in Fig. 9. The vehicle control device 100 may be, for example, an ECU (Electronic Control Unit), an OBU (On Board Unit), or an external device installed near the dashboard of the vehicle 1.

[0134] The vehicle control device 100 can be configured in terms of hardware as shown in Fig. 12. Fig. 12 is a diagram showing the hardware configuration of the vehicle control device 100.

[0135] The vehicle control device 100 has an imaging IF (Interface) 101, a CPU (Central Processing Unit) 102, a RAM (Random Access Memory) 103, a ROM (Read Only Memory) 104, a lighting IF 105, a GPS (Global Positioning System) IF 106, a vehicle speed IF 107, a communication IF 108, and a bus 109. The imaging IF 101, the CPU 102, the RAM 103, the ROM 104, the lighting IF 105, the GPS IF 106, the vehicle speed IF 107, and the communication IF 108 are connected to each other via the bus 109 so as to be able to communicate with each other. A control program is stored in the ROM 104.

[0136] The imaging IF 101 is communicably connected to the imaging unit 44 via a cable or the like. The imaging IF 101 acquires an image captured by the imaging unit 44. The imaging IF 101 continuously acquires a plurality of frame images of a moving image captured by the imaging unit 44.

[0137] The lighting IF105 is communicably connected to each of the light emission control circuits 53, 54 in the lighting unit 45 via a CAN or a cable or the like. The lighting IF105 can supply control signals to each of the light emission control circuits 53, 54. This allows the lighting IF105 to control the lighting state of each of the light sources D1 to D15 in the light-emitting unit groups 51, 52.

[0138] The GPSIF 106 is communicatively connected to the GPS sensor 47 via a CAN or a cable. The GPS sensor 47 receives a GPS signal. The GPSIF 106 acquires the GPS signal received by the GPS sensor 47 and generates position information such as the latitude, longitude, and altitude of the vehicle 1 based on the GPS signal.

[0139] The vehicle speed IF 107 is communicatively connected to the vehicle speed sensor 48 via a CAN or a cable or the like. The vehicle speed sensor 48 is disposed near the wheels 42, and generates a vehicle speed pulse indicating the rotation speed or number of rotations of the wheels 42. The vehicle speed IF 107 acquires the vehicle speed pulse generated by the vehicle speed sensor 48, and calculates the traveling speed of the vehicle 1 based on the vehicle speed pulse.

[0140] The communication IF 108 is communicatively connected to the communication device 9 via a CAN or a cable or the like. The communication device 9 communicates with a server device (not shown) mainly via a wireless communication line and can receive predetermined information from the server device. The communication IF 108 can obtain predetermined information from the server device via the communication device 9.

[0141] The GPS sensor 47 and the vehicle speed sensor 48 are used to detect the position and amount of movement of the vehicle, and can function as a movement detection unit configured to detect movement in a predetermined traveling direction of the vehicle body 43. The movement detection unit may be replaced by other means capable of detecting movement of the vehicle body 43 in a predetermined traveling direction and having a similar effect.

[0142] The vehicle control device 100 may be functionally configured as shown in Fig. 13. Fig. 13 is a diagram showing the functional configuration of the vehicle control device 100. The CPU 102 reads out a control program from the ROM 104, and expands each of the functional configurations shown in Fig. 13 in a buffer area of ​​the RAM 103 either collectively when compiling the control program or sequentially according to the progress of processing by the control program.

[0143] The vehicle control device 100 has an acquisition unit 110, an acquisition unit 120, and a control unit 130. The control unit 130 has a plurality of shift registers 131, 136, a processing unit 133, a distance calculation unit 134, and a timing control unit 135. Each of the shift registers 131, 136 has a plurality of stages of registers 132-1 to 132-50.

[0144] The acquisition unit 110 acquires images of the image acquisition areas IM1 and IM2. The acquisition unit 110 acquires, for example, an entire image IM as shown in FIG. 14 captured by the imaging unit 44. FIG. 14 is a diagram showing positions IM1 and IM2 of the image acquisition areas in the entire image IM. FIG. 14 illustrates an example of the entire image IM and positions IM1 and IM2 of the image acquisition areas relative to the forward view viewed through the window 43b from inside the vehicle compartment 46 corresponding to FIG. 10. The acquisition unit 110 cuts out the image acquisition areas IM1 and IM2 from the acquired entire image IM. The positions of the image acquisition areas IM1 and IM2 to be cut out in the entire image IM are determined in advance and set in the control program.

[0145] The position of image acquisition area IM1 in the overall image IM is a position corresponding to the arrangement direction of multiple light-emitting elements GR1-GR3 of light-emitting element group 51 arranged on the ceiling of vehicle interior 46, as shown by dotted lines in Fig. 14, for example, a position on the +Y side to +Z side from the center of the overall image IM. The position of image acquisition area IM2 in the overall image IM is a position corresponding to the arrangement direction of multiple light-emitting elements GR1-GR3 of light-emitting element group 52 arranged on the ceiling of vehicle interior 46, as shown by dotted lines in Fig. 14, for example, a position on the -Y side to +Z side from the center of the overall image IM.

[0146] 13 acquires the traveling speed of the vehicle 1 calculated by the vehicle speed IF 107. The acquisition unit 120 supplies the traveling speed of the vehicle 1 to a distance calculation unit 134 of the control unit 130.

[0147] The distance calculation unit 134 obtains the travel distance of the vehicle 1 by integrating the travel speed, etc. The distance calculation unit 134 supplies the travel distance of the vehicle 1 to the timing control unit 135.

[0148] The multiple shift registers 131, 136 correspond to the multiple image acquisition areas IM1, IM2, the multiple light emission control circuits 53, 54, and the multiple light-emitting unit groups 51, 52. The shift register 131 has multiple stages of registers 132-1 to 132-50. Each of the shift registers 131, 136 receives and holds an image of the corresponding image acquisition area IM1, IM2, and shifts the image held in the register 132 to the next stage register 132 at a timing controlled by the timing control unit 135. In the following, the shift register 131, the image acquisition area IM1, the light emission control circuit 53, and the light-emitting unit group 51 will be mainly described as examples, but the same applies to the shift register 136, the image acquisition area IM2, the light emission control circuit 54, and the light-emitting unit group 52.

[0149] The timing control unit 135 sequentially obtains a plurality of timings based on a point in the image acquisition area IM1 according to the travel distance of the vehicle 1. As shown in FIG. 15, the plurality of timings are obtained by dividing the distance L IM Each timing corresponds to a plurality of points obtained by dividing the image acquisition area IM1 by a predetermined interval ΔL. Each timing is the timing when the vehicle 1 passes through the corresponding point. FIG. 15 is a diagram showing spatial changes of the plurality of light-emitting elements GR1 to GR3 according to images at the plurality of timings. In FIG. 15, the distance L IM This illustrates the case where =ΔL×49.

[0150] If the road is undulating or curved, the captured image may differ from the actual location of the road. IM For example, the distance L may be set to a distance at which it takes 2 to 3 seconds from when the image acquisition area IM1 is acquired to when the vehicle passes the point of the image acquisition area IM1. IM may be, for example, about several tens of meters. In addition, the predetermined interval ΔL and the number of divisions can be arbitrarily determined depending on the traveling speed of the vehicle 1, the specifications of the imaging unit 44, and the like.

[0151] The multiple timings to be determined by timing control section 135 correspond to multiple points spaced at predetermined distances along which vehicle 1 should travel.

[0152] In the case of Figure 15, the X position of the image acquisition area IM1 is x 49 The X position of the image capturing unit 44 of the vehicle 1 is x0. If the position of the vehicle 1 is expressed by the X position of the image capturing unit 44, the vehicle 1 travels in the X direction and passes through multiple points x0, x1, x2, . . . , x 49 Pass through each point x0, x1, x2,..., x 49 corresponds to a number of points arranged in the X direction at a predetermined interval ΔL. 49 The timing of passing through is t0, t1, t2, . . . , t 49 Let us assume that.

[0153] Each of the multiple timings is the timing at which an image acquired by the acquisition unit 110 shown in Fig. 13 should be stored in the shift register 131. The timing control unit 135 supplies the obtained timings to the acquisition unit 110. The acquisition unit 110 continuously acquires multiple frame images of a moving image captured by the imaging unit 44, and selects the frame images acquired from the imaging unit 44 at the timings controlled by the timing control unit 135 and stores them in the first-stage register 132 of the shift register 131.

[0154] In the case of FIG. 15, at timing t0, the acquisition section 110 selects a frame image as the entire image IM, cuts out an image FR_49 of an image acquisition area IM1 from the frame image, and stores the image FR_49 in a first-stage register 132 of a shift register 131.

[0155] Each of the multiple timings is a timing at which the shift register 131 shown in Fig. 13 shifts an image stored in the register 132 at each stage. The timing control unit 135 supplies the obtained timing to the register 132 at each stage of the shift register 131. The registers 132-1 to 132-49 at each stage transfer the image to the register 132 at the next stage and the processing unit 133 at the timing controlled by the timing control unit 135. The register 132-50 at the final stage supplies the image to the processing unit 133 at the timing controlled by the timing control unit 135.

[0156] 15, at timing t0, the shift register 131 transfers the images FR_48 to FR_0 of the image acquisition area IM1 stored in the registers 132-1 to 132-49 in each stage to the next stage register 132. In FIG. 15, the transferred state is illustrated.

[0157] Each of the multiple timings is a timing for performing image processing in processing unit 133 shown in Fig. 13. Processing unit 133 acquires images from registers 132-1 to 132-50 in each stage of shift register 131 at a timing controlled by timing control unit 135, and performs statistical processing on the multiple acquired images.

[0158] Each of the light emission control circuits 53, 54 of the illumination unit 45 has a shift register 531 and a plurality of control circuits C1 to C15. The plurality of control circuits C1 to C15 correspond to a plurality of light sources D1 to D15. The shift register 531 has a plurality of stages of registers 532-1 to 532-3. The register 532-1 corresponds to the control circuits C1 to C5, the register 532-2 corresponds to the control circuits C6 to C10, and the register 532-3 corresponds to the control circuits C11 to C15. Each of the control circuits C1 to C15 is connected to a corresponding light source and turns the light source on and off or changes the brightness or color of the light source.

[0159] Based on the image acquired from the final-stage register 132-50, the processing unit 133 generates a control value for controlling the light source while taking into consideration the result of the statistical processing. The processing unit 133 stores the control value in the first-stage register 532 of the shift register 531. The control value may include a Y value indicating luminance, an R value which is a red color component value, a G value which is a green color component value, and a B value which is a blue color component value.

[0160] The vehicle interior lighting may be controlled one light source at a time, multiple light sources at a time, or all at once. When multiple light sources are controlled at a time or all at once, the previous brightness and lighting color and the brightness and color may be smoothed to complement the change so that the change is smooth. In addition, if the number of image frames is smaller than the control unit of the vehicle interior lighting, the brightness and lighting color may be complemented so that the change is smooth from the previous and next images.

[0161] 13 illustrates an example in which a plurality of light-emitting units are controlled collectively. In this case, the light-emitting unit group 51 may be grouped into a plurality of light-emitting units GR1 to GR3. The light-emitting unit GR1 includes light sources D1 to D5, the light-emitting unit GR2 includes light sources D6 to D10, and the light-emitting unit GR3 includes light sources D11 to D15. The light-emitting unit GR1 corresponds to the register 532-1 and the control circuits C1 to C5. The light-emitting unit GR2 corresponds to the register 532-2 and the control circuits C6 to C10. The light-emitting unit GR3 corresponds to the register 532-3 and the control circuits C11 to C15.

[0162] The control value stored in the first-stage register 532-1 of the shift register 531 is supplied to the control circuits C1 to C5 corresponding to the light-emitting unit GR1. The control circuits C1 to C5 control the brightness and color of the light sources D1 to D5 according to the control value.

[0163] In the case of Fig. 15, at timing t0, the processing unit 133 performs statistical processing on a plurality of images FR_49 to FR_0, and generates a control value FR_-1 according to the results of the statistical processing and stores it in the first-stage register 532 of the shift register 531. In Fig. 15, the luminance is indicated by the density of hatching, and the lower the density of the hatching, the higher the luminance. The control value FR_-1 indicates a relatively high luminance. In response to this, the control circuits C1 to C5 light the light sources D1 to D5 of the light-emitting unit GR1 in a color according to the control value FR_-1, and at a relatively high luminance according to the control value FR_-1.

[0164] Each of the multiple timings is a timing at which the control of the light-emitting unit group 51 by the light-emission control circuit 53 shown in Fig. 13 is changed, and is a timing at which the shift register 531 shifts the control value stored in the register of each stage. The timing control unit 135 supplies the obtained timing to the register 532 of each stage of the shift register 531. The registers 532-1 to 532-3 of each stage transfer the control value to the register 532 of the next stage at the timing controlled by the timing control unit 135. The register 532-3 of the final stage discards the control value at the timing controlled by the timing control unit 135.

[0165] In the case of FIG. 15, at timing t0, the shift register 531 transfers the control values ​​FR_-2 to FR_-3 stored in the registers 532-1 to 532-2 of the respective stages to the register 132 of the next stage, and discards the control value FR_-4 stored in the register 532-3 of the final stage. In FIG. 15, the transferred state is illustrated. In response to this, the control circuits C6 to C10 light the light sources D6 to D10 of the light-emitting unit GR2 in a color corresponding to the control value FR_-2, and at a relatively dark brightness corresponding to the control value FR_-2. The control circuits C11 to C15 light the light sources D11 to D15 of the light-emitting unit GR3 in a color corresponding to the control value FR_-3, and at an intermediate brightness corresponding to the control value FR_-3.

[0166] As shown in FIG. 15, when multiple light sources correspond to multiple points, the lighting state of the light source corresponding to each point is controlled to match the attribute of the image captured at that point. -1 The lighting state of the light sources D1 to D5 of the light-emitting part GR1 corresponding to the point x -1 The image is controlled to match the attributes of the image FR_-1 of the image acquisition area IM1 acquired at point x -2 The lighting state of the light sources D6 to D10 of the light-emitting part GR2 corresponding to point x -2 The image acquisition area IM1 is controlled to match the attributes of the image FR_-2 acquired at point x -3 The lighting state of the light sources D11 to D15 of the light-emitting unit GR3 corresponding to point x -3 The image acquisition area IM1 is controlled to match the attributes of the image FR_-3 acquired in the image acquisition area IM1.

[0167] For example, at timing t0, the timing point x 49 As shown in FIG. 16(a), the image FR_49 of the image acquisition area IM1 acquired at 49 The time t 49 13, the image is stored in the final-stage register 132-50 of the shift register 131 shown in Fig. 13. Fig. 16 is a diagram showing temporal changes in the light-emitting units GR1 to GR3 corresponding to images at multiple timings. At this time, the processing unit 133 acquires images from the registers 132-1 to 132-50 of each stage, and performs statistical processing on the acquired images.

[0168] As shown in FIG. 16(b), vehicle 1 is at point x 50 The time t 50 In the step S100, the processing unit 133 generates a control value FR_49 for controlling the light source based on the image acquired from the final-stage register 132-50 while taking into consideration the results of the statistical processing. The processing unit 133 stores the control value FR_49 in the first-stage register 532-1 of the shift register 531. In response to this, the control circuits C1 to C5 control the point x 49The light sources D1 to D5 of the light-emitting unit GR1 corresponding to the control value FR_49 are turned on in a color according to the control value FR_49 and at an intermediate brightness according to the control value FR_49.

[0169] As shown in FIG. 16(c), when the vehicle 1 is at the X position x 51 The time t 51 In response to this, the control circuits C6 to C10 transfer the control value FR_49 to the register 532-2 at the next stage of the shift register 531. 49 The light sources D6 to D10 of the light-emitting unit GR2 corresponding to the control value FR_49 are lit in a color according to the control value FR_49 and at an intermediate brightness according to the control value FR_49.

[0170] Although not shown in the figure, the vehicle 1 is at position x 52 The time t 52 In response to this, the control circuits C11 to C15 transfer the control value FR_49 to the register 532-3 at the next stage. 49 The light sources D11 to D15 of the light-emitting unit GR3 corresponding to the control value FR_49 are turned on in a color according to the control value FR_49 and at an intermediate brightness according to the control value FR_49.

[0171] As shown in Fig. 15 and Fig. 16, the attribute of an image captured at a certain point is converted into a light source control value when the point is reached, and the light source that reflects the control value is sequentially switched as the light source corresponding to the point among the multiple light sources sequentially changes in the direction opposite to the traveling direction. As for the image FR_98, it is captured at the timing t0 shown in Fig. 15 and at the timing t 49 The attribute is converted to the control value FR_49 at timing t 50 The control value FR_49 is reflected in the light emission control of the light sources D1 to D5 of the light-emitting unit GR1 at the timing t 51 The control value FR_49 is reflected in the light emission control of the light sources D6 to D10 of the light-emitting unit GR2, and at a timing t 52The control value FR_49 is reflected in the light emission control of the light sources D11 to D15 of the light-emitting unit GR3. As a result, the light emission states of the multiple light-emitting units GR1 to GR3 are controlled to flow in the direction opposite to the traveling direction when viewed from inside the vehicle interior 46.

[0172] The statistical processing performed by the processing unit 133 shown in FIG. 13 can be any statistical processing that allows the interior lighting to match the physical sensation of the occupants when the attributes of the captured image are reflected in the interior lighting.

[0173] It is possible to use the image of only one camera pixel per image acquisition area to control the line lighting. In this case, one pixel is acquired from a point on the extension of the dotted line in Fig. 14. In the example of Fig. 14, the tunnel lighting is not captured on the extension of the dotted line, so the one pixel is almost dark and the line lighting becomes dark. As vehicle 1 travels, it passes through the tunnel lighting. In other words, it is necessary to reflect the tunnel lighting to match the bodily sensation.

[0174] In order to match the bodily sensation, the image acquisition area is set to a horizontally long ellipse or a horizontally long rectangle with a point on the dotted line in FIG. 14 as the center. FIG. 14 illustrates an example in which the image acquisition area is set to a horizontally long rectangle. The image in the image acquisition area includes a plurality of pixel signals arranged two-dimensionally in a matrix. Therefore, as shown in FIG. 17, the image acquisition area IM1 can be divided two-dimensionally into a plurality of divided regions DV1 to DV25, and each divided region DV1 to DV25 can include one or more pixel signals. FIG. 17 illustrates an example in which the image acquisition area IM1 is divided vertically and horizontally into 5 regions to form 25 regions. FIG. 17 illustrates the image acquisition area IM1, but the same is true for the image acquisition area IM2.

[0175] Each of the pixel signals included in the image includes, as attributes, a Y signal indicating luminance, an R signal indicating a red (R) color component value, a G signal indicating a green (G) color component value, and a B signal indicating a blue (B) color component value. The processing unit 133 averages the luminance of the divided regions DV1 to DV25 of the image to be processed to obtain a luminance AVEY, and averages the red (R), green (G), and blue (B) color component values ​​of the divided regions DV1 to DV25 to obtain a color AVERGB. The processing unit 133 identifies the region with the maximum luminance (e.g., DV18) among the divided regions DV1 to DV25 of the image to be processed, and sets the attribute of the region as a representative value A. The representative value A includes the luminance AY of the region with the maximum luminance, and the color ARGB, which is the red (R), green (G), and blue (B) color component value of the region with the maximum luminance. The processing unit 133 identifies the area with the lowest brightness (for example, DV3) among the multiple divided areas DV1 to DV25 of the image to be processed, and sets the attribute of the area as a representative value B. The representative value B includes the brightness BY of the area with the lowest brightness, and the colors BRGB which are the red (R), green (G), and blue (B) color component values ​​of the area with the lowest brightness. Note that when the emission color of each of the light sources D1 to D15 is white, the attributes AVERGB, ARGB, and BRGB which indicate the color component values ​​may be omitted.

[0176] Images from the image acquisition area are accumulated in a shift register as the car moves along and as time passes, and are reflected in the row lighting control. This process obtains brightness and color information for representative values ​​A and B, and the same evaluation is performed again when the next image is acquired to obtain the areas for representative values ​​A and B for the next image. By defining representative value A over a wide image acquisition area, it is possible to express point-like bright spots, such as tunnel lighting.

[0177] Using representative value A to turn on the row lights works well in dark places, but there is a concern that during the day, for example, the representative value A will reflect the bright sky, resulting in a dull, almost pure white appearance. To address this issue, representative value B is used.

[0178] To match the human experience, feature points of brightness are extracted from the image of the image acquisition area, and the interior lighting is controlled to match the brightness of the feature points. If there are no feature points in the image of the image acquisition area, the interior lighting is controlled using the average brightness.

[0179] As shown in FIG. 18, the processing unit 133 may hold three types of attributes of the image to be processed: an average value AVE (AVEY, AVERGB), a representative value A (AY, ARGB), and a representative value B (BY, BRGB). FIG. 18 is a diagram showing a process for determining the luminance of a light source. In FIG. 18, the vertical axis indicates the luminance level, and the horizontal axis indicates time. The processing unit 133 obtains a reference luminance AVEY50 by averaging the average value AVEY of a plurality of images, for example, the average value AVEY of the latest 50 images, as a luminance standard. As shown in FIG. 18(a) to FIG. 18(f), if the image acquisition area is dark all the time, AVEY50 is small because the luminance is low, and if it is bright all the time, it is large. The reference luminance AVEY50 indicates a luminance standard that reflects the latest luminance trend.

[0180] At timing TM1 shown in FIG. 18(a), for example, the vehicle 1 is traveling in a tunnel or in the dark. The difference between the AY and BY of the image to be processed and the reference luminance AVEY50, which is lower than a predetermined value, is within a predetermined threshold range, and the AY and BY of the image to be processed are close. This indicates that the image is uniformly dark and has no feature points. For this reason, the processing unit 133 generates a control value for the row lights based on AVE, as shown by the dotted circle. Depending on AVE, a control value instructing to turn off the lights may also be generated.

[0181] At timing TM2 shown in FIG. 18(b), for example, the vehicle 1 is illuminated by the tunnel lighting. The difference between the reference luminance AVEY50, which is lower than a predetermined value, and the AY of the image to be processed is higher than a predetermined threshold range. This indicates that the image is dark overall, but has bright points. For this reason, the processing unit 133 generates a control value for controlling the lighting of the light source based on the representative value A(AY, ARGB), as shown by the dotted circle.

[0182] At timing TM3 shown in FIG. 18(c), for example, the vehicle 1 is about to exit a tunnel. The difference between the BY of the image to be processed and the reference luminance AVEY50, which is higher than a predetermined value, is below a predetermined threshold range and low. This indicates that the image is bright overall, but there are dark spots. For this reason, the processing unit 133 generates a control value for controlling the lighting of the light source based on the representative value B(BY, BRGB), as shown by the dotted circle. This expresses a "dark spot in a bright spot."

[0183] At timing TM4 shown in FIG. 18(d), for example, the vehicle 1 exits a tunnel and passes through a uniform sky. The difference between the AY and BY of the image to be processed and the reference luminance AVEY50, which is higher than a predetermined value, is within a predetermined threshold range, and the AY and BY of the image to be processed are close. This indicates that the image is uniformly bright and has no feature points. For this reason, the processing unit 133 generates a control value for controlling the lighting of the light source based on the average value AVE(AVEY, AVERGB), as shown by the dotted circle. However, the processing unit 133 may cap the value of the average luminance at an intermediate level of luminance regardless of AVEY.

[0184] At timing TM5 shown in FIG. 18(e), for example, the vehicle 1 is traveling near a building. The difference between the reference luminance AVEY50, which is higher than a predetermined value, and the AY of the image to be processed is higher than a predetermined threshold range. This indicates that the image is bright overall, but there are even brighter points. For this reason, the processing unit 133 generates a control value for controlling the lighting of the light source based on the representative value A(AY, ARGB), as shown by the dotted circle.

[0185] At timing TM6 shown in FIG. 18(f), for example, the vehicle 1 is traveling through sunlight filtering through the trees. For a reference luminance AVEY50 that is higher than a predetermined value, the difference between the AY of the image to be processed is higher than a predetermined threshold range, and the difference between the BY of the image to be processed is lower than a predetermined threshold range. This indicates that the image is bright overall, but bright and dark points are scattered throughout. For this reason, the processing unit 133 generates a control value for controlling the lighting of the light source based on the representative value A(AY, ARGB), as indicated by the dotted circle.

[0186] As shown in Figs. 18(a) to 18(f), in various external environments where there are changes in luminance, the linear lighting is controlled to give the appearance of movement.

[0187] Next, the flow of operation of the vehicle 1 will be described with reference to Fig. 19 and Fig. 20. Fig. 19 and Fig. 20 are flowcharts showing the operation of the vehicle 1. Fig. 19 and Fig. 20 illustrate the processing for each row of lighting. When there are multiple rows of lighting and there are m image acquisition areas, the processing shown in Fig. 19 and Fig. 20 may be performed in parallel for m areas.

[0188] When the vehicle control device 100 receives a start request from a passenger, it performs initial setting assuming that driving of the vehicle 1 will start (S101). The vehicle control device 100 initializes each parameter AVEY(*), AY(*), BY(*), AVERGB(*), ARGB(*), and BRGB(*) to 0. A number for identifying an image is placed in parentheses in the parameter name, and the smaller the number, the older the parameter data. The vehicle control device 100 initializes the registers 132, 532 in each stage of the shift registers 131, 531 to 0. At this time, the vehicle control device 100 does not cause each of the light-emitting elements GR1 to GR3 of the light-emitting element groups 51, 52 to emit light, but may perform control to emit light like a welcome light when getting on and off.

[0189] The vehicle control device 100 checks the mode setting request for the light emission control of the light-emitting unit groups 51 and 52, and if setting to the light flow mode is not requested (No in S102), returns the process to S101.

[0190] If a request to set the light flow mode has been received (Yes in S102), the vehicle control device 100 captures an image of the image acquisition area ahead via the imaging unit 44 (S103).

[0191] The vehicle control device 100 searches for the highest brightness point A and the lowest brightness point B in the captured image. When the highest brightness point A and the lowest brightness point B are found, the vehicle control device 100 calculates the brightness AY of the highest brightness point A and the ratio ARGB of the color component values ​​(S104).

[0192] If the latest image is image No. 50, the vehicle control device 100 writes information about the maximum luminance point A of the 50th image to AY(50), ARGB(50), and writes information about the minimum luminance point B of the 50th image to BY(50), BRGB(50) (S105).

[0193] The vehicle control device 100 calculates the average brightness AVEY and the average RGB ratio AVERGB of the 50th image, and writes them to AVEY(50) and AVERGB(50), respectively (S106).

[0194] The vehicle control device 100 averages the average image brightness for images 1 to 50 as a brightness standard, and updates the reference brightness AVEY50 (S107, B).

[0195] Note that n=0 is the data used to light the front of the row of lights. When vehicle 1 passes the location of the data, the data is discarded.

[0196] The vehicle control device 100 shifts the light emission control value of the entire light-emitting unit group by one in the direction opposite to the traveling direction when viewed from inside the vehicle compartment 46 (S108). Although the light emission control value is expressed as being shifted by one, it does not necessarily mean the control value of one physical light source, but may be the control value for each control unit, i.e., for each light-emitting unit. An averaging process of the control values ​​may be performed between multiple control units to smooth the variation in color and light amount between the front and rear control units.

[0197] The vehicle control device 100 compares the difference (AY(0)-AVEY50) from the maximum reference luminance for image 0 with the reference value K1. If the difference (AY(0)-AVEY50) from the maximum reference luminance exceeds the reference value K1 (Yes in S109), the vehicle control device 100 determines that point A should be reflected in the lighting control, and controls the lighting of the leading light source using AY(0), ARGB(0) (S110).

[0198] If the difference from the reference luminance of the highest luminance (AY(0)-AVEY50) is equal to or less than the reference value K1 (No in S109), the vehicle control device 100 compares the difference from the reference luminance of the lowest luminance (AVEY50-BY(0)) for image 0 with the reference value K2. If the difference from the reference luminance of the lowest luminance (AVEY50-BY(0)) is below the reference value K2 (Yes in S111), the vehicle control device 100 determines that point B should be reflected in the light emission control, and controls the lighting of the leading light source using BY(0), BRGB(0) (S112).

[0199] If the difference (AVEY50-BY(0)) from the minimum brightness reference brightness is equal to or greater than the reference value K2 (No in S111), the vehicle control device 100 determines that the average value should be reflected in the light emission control, and controls the light emission of the leading light source using AVEY(0), AVERGB(0) (S113).

[0200] The vehicle control device 100 waits until the vehicle 1 travels a certain distance ΔL from the point in S103 (No in S114), and when the certain distance ΔL has been traveled (Yes in S114), shifts the register in each stage of the shift register by one. That is, n is decremented by 1 in the parameters AVEY(n), AY(n), BY(n), AVERGB(n), ARGB(n), and BRGB(n) (S115). By linking the physical shift amount of the light-emitting unit per time by the "certain distance" traveled and the shift in the light-emitting control unit, it is possible to make the light flow smoothly in conjunction with changes in the surrounding light.

[0201] Thereafter, the vehicle control device 100 returns the process to S102 (A) and repeats the processes from S2 onwards.

[0202] As described above, in the second embodiment, in the vehicle 1, a plurality of light-emitting units GR1 to GR3 are arranged in a row on the ceiling of the vehicle interior 46 along the traveling direction, and the light-emitting state of each of the plurality of light-emitting units GR1 to GR3 is controlled according to an image captured in front of the vehicle body 43. This makes it possible to realize an illumination environment conforming to the scenery above on the ceiling of the vehicle interior 46 in real time, thereby enhancing the sense of speed, immersion, and realism when the vehicle 1 is traveling. Therefore, the entertainment value inside the vehicle interior 46 can be improved.

[0203] Note that, as long as the imaging unit 44 can capture images of the image acquisition areas IM1 and IM2, it may be disposed in a position other than the +X side and +Z side in the vehicle interior 46, or may be disposed outside the vehicle interior 46. Furthermore, the imaging unit 44 may also be a camera disposed in the vehicle interior 46 for another purpose, for example, a camera for a drive recorder or an ADAS (Advanced Driver Assistance Systems).

[0204] Furthermore, the number of light-emitting unit groups included in the illumination unit 45 may be one or three or more. When the illumination unit 45 has three or more light-emitting unit groups, the imaging unit 44 may be capable of capturing images of three or more image acquisition areas corresponding to the three or more light-emitting unit groups. The imaging unit 44 may have one camera 44a capable of capturing images of the three or more image acquisition areas, or may have three or more cameras 44a capable of capturing images of the three or more image acquisition areas, respectively. The three or more cameras 44a may be lined up along the Y direction at positions on the +X side and +Z side in the vehicle interior 46.

[0205] As a first modified example of the second embodiment, if the vehicle 1 can obtain map information related to objects along the road ahead, the map information may be further reflected in lighting control within the vehicle cabin 46. The map information is information in which the geographical positions of objects and attributes such as color and brightness of the objects along the road are associated with each other for a plurality of objects.

[0206] In the vehicle 1, the vehicle control device 100 receives map information from a cloud server or the like via the communication device 9 shown in FIG. 12 at a predetermined timing, and temporarily stores the map information in the RAM 103 (see FIG. 12) via the communication IF 108. The vehicle control device 100 refers to the map information to grasp the geographical positions of objects. The vehicle 1 captures images of objects along the road along which the vehicle 1 will proceed, in the image acquisition area, with the imaging unit 44. The objects along the road may be fixed objects along the road, such as trees, historical buildings, tunnel walls, and street lamps. The map information may be stored in advance in the ROM 104 (see FIG. 12) instead of being received and acquired from a cloud server or the like.

[0207] In the vehicle 1, the vehicle control device 100 sequentially receives GPS signals via the GPS sensor 47, sequentially obtains the current position of the vehicle 1 via the GPS IF 106, and sequentially corrects the current position with the vehicle speed obtained via the vehicle speed IF 107. When generating the control values ​​of the light-emitting unit groups 51 and 52 corresponding to the images captured in the image acquisition areas IM1 and IM2, the vehicle control device 100 makes changes according to the map information. When the vehicle control device 100 determines that the vehicle 1 has reached the geographical position of an object, it identifies attributes such as color and brightness of the object corresponding to the geographical position in the map information. The vehicle control device 100 changes the control values ​​of the light-emitting unit groups 51 and 52 according to the captured image according to the attributes identified from the map information. This makes it possible to make the light-emitting control of each of the light-emitting units GR1 to GR3 of the light-emitting unit groups 51 and 52 more suitable for the objects along the road, and further enhances the sense of immersion and realism of the traveling of the vehicle 1.

[0208] The vehicle control device 100 may reflect the characteristic color of the surrounding scenery inside the vehicle. For example, if the object is a tree, the vehicle control device 100 controls the light emitting color of the light emitting units GR1 to GR3 to be green or the color of autumn leaves. If the object is a streetscape or a historical building, the vehicle control device 100 controls the light emitting color of the light emitting units GR1 to GR3 to be the color of a wall or roof. When the vehicle 1 enters a tunnel, the vehicle control device 100 darkens the brightness of the light emitting units GR1 to GR3, and when the vehicle 1 passes through a street light after it gets dark, the vehicle control device 100 controls the brightness of the light emitting units GR1 to GR3 to be brighter.

[0209] In the vehicle 1, the vehicle control device 100 may determine, for each object on the road, whether the control values ​​of the light emitting unit groups 51 and 52 generated from the captured image of the imaging unit 44 are used as they are or whether the control values ​​are changed according to map information. The object to be used as is is a tree, and the object to be changed in control value is a tunnel, a street lamp, a streetscape, a historical building, etc. The vehicle 1 may change the attributes specified from the map information successively according to the weather, the date and time, the time period, and an event, so that various responses are possible, such as giving a sense of the season or changing the color depending on the time period through which the vehicle passes. The vehicle control device 100 can acquire map information of the road, for example, the season and weather in which trees with red leaves can be seen from the car window on that road. The vehicle 1 can not only capture the brightness and color of the image acquisition area with the camera of the imaging unit 44, but also detect the trees from the car window as objects, and can see that the trees are all uniformly red-leaved from the map information of the road, so that red and yellow flow inside the car according to the position of the trees from the car window. Similarly, Car 1 can have a flowing blue and white display if snow-covered trees and blue skies are visible from the window.

[0210] At this time, the control of brightness etc. according to the map information may be combined with the control of brightness etc. as shown in Figs. 17 to 20. AVEY and AVERGB may be the same as those in Figs. 17 to 20, and AY, ARGB and BY, BRGB may be replaced with colors according to attributes identified from the map information. The control of brightness according to the map information may be performed, for example, as shown in Fig. 21. Fig. 21 is a diagram showing the control of a plurality of light-emitting units according to the image and map information of the image acquisition area in a first modified example of the second embodiment.

[0211] The vehicle control device 100 changes the values ​​of AY and BY identified from the captured image so that they are sufficiently larger than the reference values ​​K1 and K2, so that the colors shown as high luminance point A and low luminance point B are reflected in the light emission control of the light-emitting element groups 51 and 52.

[0212] In the case shown in Fig. 21, when vehicle 1 reaches the "streetlight area", it combines lighting control based on captured images with lighting control based on map information + GPS. For the four points in the "streetlight area", vehicle 1 sets the control values ​​for high brightness point A to none, none, blue-white, none, and the control values ​​for low brightness point B to black, black, none, black. "None" indicates that there is no control value, so the value of high brightness point A or low brightness point B acquired by the camera is used as is.

[0213] When vehicle 1 reaches the "tunnel area", it controls the lighting based on map information and GPS. For the three points in the "tunnel area", vehicle 1 sets the control values ​​for high luminance point A to none, none, none, and the control values ​​for low luminance point B to black, black, black.

[0214] When vehicle 1 reaches the "sunset area" in the evening on a clear day, it controls the lighting based on map information, GPS, weather, date, time, and time. For the three points in the "tunnel area," vehicle 1 sets the control values ​​for high-luminance point A to dark blue, purple, and orange, and the control values ​​for low-luminance point B to none, none, and none.

[0215] When vehicle 1 reaches the "historic building area" during the daytime, it controls the lighting based on map information, GPS, and time of day. For the three points in the "historic building area," vehicle 1 sets the control values ​​for high-luminance point A to white, white, and white, and the control values ​​for low-luminance point B to none, none, and none.

[0216] When vehicle 1 reaches the "tree-lined road area" during the daytime, it combines lighting control based on captured images with lighting control based on map information + GPS. For the five points in the "tree-lined road area," vehicle 1 sets the control values ​​for high brightness point A to green, green, green, green, green, and sets the control values ​​for low brightness point B to none, none, none, none, none.

[0217] The vehicle control device 100 can update object information in the map information received from a cloud server or the like via the communication device 9 with information on the object detected during actual driving, and can reflect this in the light emission control of the light emitting unit groups 51, 52 when driving along the road on which the object is later detected. For example, in cases where the number of objects of a different color increases or decreases or their positions change due to construction work, or the visible color of an object changes due to weather changes, the vehicle control device 100 can update the map information to a more appropriate color for the object.

[0218] Furthermore, as shown in Fig. 22, the vehicle 1 may operate differently from the second embodiment in the following respects: Fig. 22 is a flowchart showing the operation of the vehicle 1 according to a first modified example of the second embodiment.

[0219] After the processes of S1 to S7 are performed, the vehicle control device 100 receives a GPS signal via the GPS sensor 47, obtains the current position of the vehicle 1 by the GPS IF 106, and corrects the current position with the vehicle speed obtained by the vehicle speed IF 107 (S121).

[0220] The vehicle control device 100 refers to the map information, and if available, changes AY, ARGB, BY, and BRGB (S122, B). That is, when the vehicle control device 100 determines that the vehicle 1 has reached the geographical position of an object, it identifies attributes such as color and brightness of the object corresponding to that geographical position in the map information. The vehicle 1 overwrites AY, ARGB, BY, and BRGB with AY, ARGB, BY, and BRGB according to the attributes identified from the map information. Then, the vehicle control device 100 performs the processes from S108 onwards.

[0221] It should be noted that the vehicle control device 100 may skip the processes of S121 and S122 when there is no map information, or when the map information is not valid due to season, time period, or weather factors.

[0222] In this way, in the first modified example of the second embodiment, the control values ​​of the light-emitting unit groups 51 and 52 based on the captured image are changed according to the attributes identified from the map information. This allows the light-emitting control of the light-emitting unit groups 51 and 52 to be more suited to the objects along the road, and further enhances the immersive and realistic feeling of the traveling of the vehicle 1.

[0223] Also, as a second modified example of the second embodiment, in the steady state, the light emission control of the light emitting unit groups 51 and 52 may be paused, and the light emission control of the light emitting unit groups 51 and 52 may be started when the geographical position of the vehicle 1 is within the attraction point. The geographical position within the attraction point may be included in the map information. The vehicle control device 100 may acquire information on the geographical position within the attraction point by cooperating with the navigation system of the vehicle 1. For example, when the vehicle control device 100 detects that the tourist spot guide function of the navigation system is turned ON, it may start the light emission control of the light emitting unit groups 51 and 52 as the vehicle is within the attraction point. Alternatively, the vehicle control device 100 may start the light emission control of the light emitting unit groups 51 and 52 in synchronization with a start signal of the tourist commentary audio of the navigation system. That is, the vehicle control device 100 may start the light emission control of the light emitting unit groups 51 and 52 as the vehicle is within the attraction point in response to the start signal of the tourist commentary audio of the navigation system becoming active.

[0224] This clarifies the location where the vehicle 1 should control the light emission. That is, the vehicle 1 keeps each of the light emitting units GR1 to GR3 non-emitted until it reaches the scenic spot, and then controls the light emission of each of the light emitting units GR1 to GR3 when it reaches the scenic spot. This creates effects such as surprising passengers when the light is turned on and attracting their interest in the explanation of the location.

[0225] As a second modification of the second embodiment, the vehicle 1 may operate in a manner different from that of the second embodiment in the following respects, as shown in Fig. 23. Fig. 23 is a flowchart showing the operation of the vehicle 1 according to the second modification of the second embodiment.

[0226] If a request to set the light flow mode has been made (Yes in S102), the vehicle control device 100 waits until the geographical position of the vehicle 1 is within the highlight point (No in S131). When the geographical position of the vehicle 1 is within the highlight point (Yes in S131), the vehicle control device 100 performs processing from S103 onwards. Although not shown, the vehicle control device 100 may determine whether the vehicle 1 has left the highlight point after S15. If the vehicle 1 is within the highlight point, the vehicle control device 100 may continue the light emission control of the light-emitting unit groups 51 and 52, and when the vehicle 1 has left the highlight point, the vehicle control device 100 may halt the light emission control of the light-emitting unit groups 51 and 52. Then (A), the vehicle control device 100 may return the processing to S2.

[0227] In this way, in the second modified example of the second embodiment, the control value of the light-emitting unit group 51, 52 based on the captured image is started in response to the geographical position of the vehicle 1 being within a scenic spot. This makes it possible to create a sense of surprise for the occupants when the light is turned on, and further enhances the sense of immersion and realism in the driving of the vehicle 1.

[0228] (Third embodiment) The third embodiment will be described with reference to the drawings.

[0229] (Vehicle configuration example) 24 is a schematic diagram showing an example of a vehicle 1 including a communication control device 400 according to the third embodiment. Fig. 24(a) is a side view of the vehicle 1, and Fig. 24(b) is a top view of the vehicle 1.

[0230] 24, a vehicle 1 includes a vehicle body 2 and two pairs of wheels 3 (a pair of front tires 3f and a pair of rear tires 3r) arranged on the vehicle body 2 along the vehicle length direction (±Y direction) of the vehicle body 2. The two paired front tires 3f and the two paired rear tires 3r are each arranged along the vehicle width direction of the vehicle body 2 (±X direction perpendicular to the ±Y direction).

[0231] The vehicle 1 is equipped with a pair of door mirrors 94 at both ends of the vehicle body 2 in the ±X direction, near the front tires 3f in the ±Y direction of the vehicle body 2, and at a predetermined height position in the vehicle height direction (the ±Z direction perpendicular to the ±X and ±Y directions).

[0232] The vehicle 1 includes a plurality of seats 95a to 95f inside the vehicle body 2. The seats 95a and 95b are arranged side by side in the ±X direction near the front tire 3f. The seats 95c and 95d are arranged side by side in the ±X direction between the front tire 3f and the rear tire 3r. The seats 95e and 95f are arranged side by side in the ±X direction near the rear tire 3r. The seat 95c is arranged behind the seat 95a, the seat 95d is arranged behind the seat 95b, the seat 95e is arranged behind the seat 95c, and the seat 95f is arranged behind the seat 95d. The number and arrangement of the seats 95 included in the vehicle 1 are not limited to the example of FIG. 24.

[0233] The vehicle 1 is provided with a plurality of communication devices 96a to 96f on the inner wall of the vehicle body 2. The communication devices (96a, 96c, 96d, 96e, 96f) used for communication between passengers in the vehicle are disposed near each seat.

[0234] The communication devices (96a, 96c, 96d, 96e, 96f) are arranged such that, for example, communication device 96a is arranged in front of seats 95a and 95b, communication device 96c is arranged to the right of seat 95c, communication device 96d is arranged to the left of seat 95d, communication device 96e is arranged to the right of seat 95e, and communication device 96f is arranged to the left of seat 95f. The communication devices (96a, 96c, 96d, 96e, 96f) are used for communication between passengers.

[0235] In this specification, the end face of the vehicle body 2 on the side of the front tire 3f may be referred to as the front face. Also, the end face of the vehicle body 2 on the side of the rear tire 3r may be referred to as the rear face. Also, both end faces of the vehicle body 2 in the ±X directions may be referred to as side faces. Also, when seated in any of the seats 95a to 95f in the vehicle 1, the side facing the right side may be referred to as the right side, and the side facing the left side may be referred to as the left side.

[0236] In this specification, the direction toward the left side of the vehicle body 2 is the +X direction, and the direction toward the right side is the -X direction. The direction toward the front side of the vehicle body 2 is the +Y direction, and the direction toward the rear side is the -Y direction. The direction toward the top of the vehicle body 2 is the +Z direction, and the direction toward the bottom (road surface side) is the -Z direction.

[0237] In addition, in this specification, when vehicle 1 is parked on a road surface having an ideal plane, the ±X axis (X axis) and ±Y axis (Y axis) of vehicle 1 are parallel to the road surface, and the ±Z axis (Z axis) of vehicle 1 is parallel to the normal to the road surface.

[0238] The vehicle 1 can travel using two pairs of wheels 3 arranged along the ±Y direction. In this case, the ±Y direction in which the two pairs of wheels 3 are arranged becomes the traveling direction (movement direction) of the vehicle 1, and the vehicle can move forward (travel in the +Y direction) or backward (travel in the -Y direction) by switching gears, etc. It can also turn right or left by steering.

[0239] The communication control device 400 is mounted on, for example, the vehicle 1, and controls the communication devices 96a, 96c, 96d, 96e, and 96f. The communication control device 400 performs control such as displaying various information on the communication control device 400 in response to a user input. The details will be described later.

[0240] (Example of hardware configuration of communication control device) Fig. 25 is a diagram illustrating an example of a hardware configuration of a communication control device 400 according to embodiment 3. As illustrated in Fig. 25(a), the communication control device 400 includes a CPU (Central Processing Unit) 221, a ROM (Read Only Memory) 222, a RAM (Random Access Memory) 23, an auxiliary storage device 224, an input device 225, a display device 226, an external I / F 227, and a speaker I / F 228.

[0241] The CPU 221 executes a program to comprehensively control the operation of the communication control device 400 and realize various functions of the communication control device 400. The various functions of the communication control device 400 will be described later.

[0242] The ROM 222 is a non-volatile memory and stores various data (information written during the manufacturing stage of the communication control device 400) including a program for starting up the communication control device 400. The RAM 223 is a volatile memory having a working area for the CPU 221. The auxiliary storage device 224 stores various data such as the program executed by the CPU 221. The auxiliary storage device 224 is configured, for example, by a HDD (Hard Disc Drive) or the like.

[0243] The input device 225 is a device for performing various operations by a passenger (here, a person who operates the communication device, etc.) using the communication control device 400. The input device 225 is configured, for example, with a touch panel or hardware keys.

[0244] The display device 226 is a display unit that displays various information including icons and seating charts. The display device 226 is, for example, a liquid crystal display, and the input device 225 and the display device 226 may be integrated into one unit, for example, in the form of a touch panel.

[0245] The external I / F 227 is an interface for connecting (communicating) with external devices such as the communication devices 96a, 96c, 96d, 96e, and 96f, and performs communication, for example, on a Local Interconnect Network (LIN).

[0246] The speaker I / F 228 is a device for outputting a ringtone when the transmitted icon and seat are received.

[0247] The light emitting device 229 is a light emitting unit that sequentially lights up from the seat from which the light emitting device transmitted the signal toward the communication device corresponding to the seat from which the light emitting device transmitted the signal. The light emitting device 229 is, for example, configured with an LED (Light Emitting Diode) or the like.

[0248] Next, FIG. 25(b) is a cross-sectional view showing the structure of the display device 226 of the communication control device according to the third embodiment.

[0249] Display device 226 includes at least operation surface 226A that can be touched by a finger, sheet 226B arranged along operation surface 226A on the underside of operation surface 226A, a light-emitting circuit (display circuit) 226C arranged along operation surface 226A on the underside of sheet 226B and capable of emitting visible light (predetermined light), and a touch panel circuit (detection circuit) 226D arranged along operation surface 226A and light-emitting circuit 226C between operation surface 226A and light-emitting circuit 226C and capable of detecting at least the movement of a finger on operation surface 226A.

[0250] The sheet 226B may be attached to the touch panel circuit 226D with double-sided tape or adhesive, or may be directly printed. The light-emitting circuit 226C is an organic EL (Electro Luminescence) display circuit, or a liquid crystal display circuit equipped with a backlight. The sheet 226B and the touch panel circuit 226D are transparent and transmit light emitted by the light-emitting circuit 226C. For example, an ultrasonic surface acoustic wave, resistive film, or capacitive touch panel is used for the touch panel circuit 226D.

[0251] (Example of communication control device functions) Next, Fig. 26 is a diagram showing an example of a functional configuration of a communication control device according to the third embodiment. Note that, in the example of Fig. 26, only functions related to this embodiment are illustrated, but the functions of the communication control device 400 are not limited to these. As shown in Fig. 26, the communication control device 400 has a control unit 31, a storage memory 32, and a working memory 33.

[0252] The control unit 31 has an icon display control unit 31A, an icon selection unit 31B, a seat display control unit 31C, a seat selection unit 31D, a transmission control unit 31E, a reception control unit 31F, and a light emission control unit 31G.

[0253] The icon display control unit 31A has a control function of displaying at least one icon used for occupant communication on the display unit. The at least one icon used for occupant communication refers to icon data 32A stored in the storage memory 32. In this embodiment, the icon display control unit 31A may have a function of starting up to display the icon data 32A used for communication on the display unit. Regarding the icon data 32A used for communication, the icon data 32A may be set as an initial setting when the icon data 32A is displayed on the display unit after the communication control device 400 is started up.

[0254] Here, an example of an icon is shown in Fig. 27(a). The upper part of the table describes what the icon represents, and the lower part shows an icon that represents the meaning of the upper part. These are merely examples, and the icons disclosed in this disclosure also include icons other than those shown in Fig. 27(a).

[0255] In addition, the control function of displaying an icon on the display unit refers to a control function of displaying icon data 32A on the display unit, and displaying icon selection data 33A selected by icon selection unit 31B from the icons of icon data 32A displayed on the display unit in response to user input.

[0256] The icon selection unit 31B has a function of selecting one icon from at least one icon in response to a user input. When the user wants to transmit an icon to another passenger, the user selects one icon from at least one icon. The selected icon is icon data 32A stored in the storage memory 32, and the selected icon is stored in icon selection data 33A stored in the working memory 33.

[0257] The seat display control unit 31C has a control function of displaying a seat chart showing the seat positions on the display unit. The seat chart showing the seat positions is the seat chart 32C stored in the storage memory 32. Here, FIG. 27(b) shows an example of a seat chart showing the seat positions. A seat chart consists of seat rows and seat data, and here is an example of a vehicle with three rows. These are merely examples, and the seat chart of the present disclosure also covers seat charts other than that of FIG. 27(b).

[0258] The seat selection unit 31D has a function of selecting a seat from the seating chart to which the icon selected by the icon selection unit is to be sent in response to a user's input. The seat to which the icon is to be sent is the seat data 32B stored in the storage memory 32, and the selected seat is stored in the seat selection data 33B stored in the working memory 33. In addition, the seat from which the user sends is stored in the seat selection data 33B.

[0259] The transmission control unit 31E has a control function of transmitting an icon to the communication device corresponding to the seat selected by the seat selection unit 31D. The selected seat is the seat selection data 33B stored in the working memory 33, and the icon to be transmitted is the icon selection data 33A stored in the working memory 33.

[0260] The reception control unit 31F has a control function of receiving an icon to a communication device corresponding to a seat transmitted by the transmission control unit 31E. The transmitted seat is the seat selection data 33B stored in the working memory 33, and the received icon is the icon selection data 33A stored in the working memory 33.

[0261] The light emission control unit 31G has a light emission control function of sequentially lighting the light emitting unit toward the communication device corresponding to the seat transmitted by the transmission control unit 31E so as to approach from the transmitted seat. The transmitted seat and the transmitted seat are seat selection data 33B stored in the working memory 33, and the sequential lighting is controlled by the light emission on / off data 33D stored in the working memory 33.

[0262] Next, the procedure of the icon transmission process executed by the control unit 230 will be described.

[0263] (Example of transmission process of communication control device) FIG. 28 is a flowchart showing an example of a procedure of a process for transmitting the icon selection data 33A executed by the control unit 230 according to the present embodiment.

[0264] First, the icon display control unit 31A and the seat display control unit 31C delete all data including the icon selection data 33A, the seat selection data 33B including the transmission / reception destination, and the seat chart data 33C (step S501). After deleting all data, the process proceeds to step S502.

[0265] In step S502, the reception control unit 31F judges whether there is received data for receiving icon selection data 33A to a communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E. If it is judged that there is received data (step S502: No), the process proceeds to reception processing S503. The function of the reception processing S503 of the control unit 230 will be described in detail later with reference to FIG. 32. If it is judged that there is no received data (step S502: Yes), the process proceeds to step S504.

[0266] In step S504, the icon display control unit 31A judges whether or not the user has touched the display unit. If it is judged that the user has not touched the display unit (step S504: No), the process returns to step S502. If it is judged that the user has touched the display unit (step S504: Yes), the process proceeds to step S505.

[0267] In step S505, in response to a user's input, the icon display control unit 31A displays the icon data 32A in order to select one icon selection data 33A from at least one icon data 32A. After displaying the icon data 32A, the process proceeds to step S506.

[0268] In step S506, it is determined whether the user wants to change the type of icon data 32A to be transmitted. If the user decides to change the type of icon data 32A (step S506: Yes), the process proceeds to step S507. In step S507, the icon display control unit 31A changes the type of icon data 32A, and the process proceeds to step S506. If the user decides not to change the type of icon data 32A (step S506: No), the process proceeds to step S508.

[0269] In step S508, the icon selection unit 31B determines whether or not one piece of icon selection data 33A has been selected from at least one piece of icon data 32A in response to the user's input. If it is determined that the user has not selected icon data 32A (step S508: No), the process proceeds to step S509.

[0270] In step S509, icon display control unit 31A determines whether a predetermined time has passed since touching the display unit. If it is determined that the predetermined time has passed (step S509: Yes), the user does not transmit icon selection data 33A, and the process of transmitting icon selection data 33A ends. If it is determined that the predetermined time has not passed (step S509: No), the process proceeds to step S505. If it is determined that the user has selected icon data 32A (step S508: Yes), the process proceeds to step S510.

[0271] In step S510, the icon display control unit 31A performs processing for erasing icon data 32A other than the one icon selection data 33A selected from at least one icon data 32A in response to a user input. When all icon data 32A other than the display of the selected icon selection data 33A has been erased, the process proceeds to step S511.

[0272] In step S511, the seat display control unit 31C controls the display unit to display the seat chart 32C indicating the seat positions. After the seat chart 32C is displayed on the display unit, the process proceeds to step S512.

[0273] In step S512, the seat selection unit 31D judges whether or not to select seat data 32B to be a destination of the icon selection data 33A selected by the icon selection unit 31B from the seat chart 32C in response to the user's input. If it is judged that the seat data 32B is not selected (step S512: No), the process proceeds to step S513.

[0274] In step S513, seat selection unit 31D determines whether a predetermined time has passed since touching the display unit. If it is determined that the predetermined time has passed (step S513: Yes), the user does not transmit icon selection data 33A, and the process of transmitting icon selection data 33A ends. If it is determined that the predetermined time has not passed (step S513: No), the process proceeds to step S512. If it is determined that seat data 32B has been selected (step S512: Yes), the process proceeds to step S514.

[0275] In step S514, the seat selection unit 31D performs processing to delete seat data 32B other than the seat selection data 33B that is the destination of the icon selection data 33A selected by the icon selection unit from the seat chart data 33C in response to the user's input. After deleting the seats other than the selected seats, the process proceeds to step S515.

[0276] In step S515, the transmission control unit 31E performs a process of transmitting the icon selection data 33A to the communication device corresponding to the seat selection data 33B selected by the seat selection unit 31D. After performing the process of transmitting the icon selection data 33A to the communication device corresponding to the selected seat selection data 33B, the process proceeds to step S516.

[0277] In step S516, the light-emission control unit 31G performs a process of causing the light-emitting units arranged between the communication devices to sequentially light up toward the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E so that the light-emitting units approach from the transmitted seat selection data 33B. When the light-emitting units perform a process of sequentially light up toward the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E so that the light-emitting units approach from the transmitted seat selection data 33B, the process of transmitting the icon selection data 33A executed by the control unit 230 is completed.

[0278] (Example of display format in transmission process of the display unit) Next, a screen transition for a user in the icon transmission process on the display unit of the communication device will be described with reference to Figs. 29, 30 and 31. Figs.

[0279] As shown in Fig. 29(a), a center display provided in an instrument panel in a vehicle will be described as an example. Screen 260 is a screen displayed on a display unit displayed in front of the front seat passengers, and the display unit will be described as a center display provided in the instrument panel of the vehicle as an example. Screen 260 may also be a screen displayed in front of the front seat passengers.

[0280] Screen 260 is made up of a liquid crystal panel and an electrostatic touch panel. Screen 260A mainly displays information related to infotainment. For example, screen 260C in screen 260A displays information related to music, screen 260D displays information related to icons 261 related to communication, and screen 260E displays information related to the vehicle.

[0281] Screen 260B mainly displays information necessary for driving, such as navigation. In addition, in the present disclosure, it is assumed that the driver is driving a right-hand drive vehicle, so in screen 260, the area of ​​screen 260A far from the driver's seat displays information related to infotainment, and screen 260B close to the driver's seat displays information necessary for driving. By displaying information necessary for driving on screen 260B, the driver's line of sight movement is reduced, and driving is not hindered.

[0282] As shown in FIG. 29(b), an icon 261 is displayed in a screen 260D (display unit), and when a user touches an icon 261 on the screen 260D (display unit), the icon display control unit 31A transitions the screen to that shown in FIG. 29(c).

[0283] As shown in FIG. 29(c), icons (262, 263, 264) are displayed on screen 260A (display unit), and in response to user input, icon display control unit 31A displays icons (262, 263, 264) (icon data 32A) on screen 260A (display unit) to select one icon (icon selection data 33A) from at least one icon (262, 263, 264) (icon data 32A).

[0284] Seats (265, 266, 267, 268) are displayed in the screen 260A (display unit), and the seat display control unit 31C displays the seats (265, 266, 267, 268) (seat chart 32C and seat data 32B) indicating the positions of the seats on the screen 260A (display unit). When the icons (262, 263, 264) (icon data 32A) and the seats (265, 266, 267, 268) (seat chart 32C and seat data 32B) are displayed on the screen 260A (display unit), a transition to FIG. 29(d) is made.

[0285] As shown in FIG. 29(d), icons (262, 263, 264) (icon data 32A) and seats (265, 266, 267, 268) (seat chart 32C and seat data 32B) are displayed on a screen 260A (display unit). An icon selection unit 31B selects one icon 263 (icon selection data 33A) from among at least one icon (262, 263, 264) (icon data 32A) in response to a user input. Thereafter, a seat selection unit 31D slides a finger toward a seat 66 (seat selection data 33B) to which the icon 263 (icon selection data 33A) selected by the icon selection unit 31B is to be sent from among the seats (265, 266, 267, 268) (seat chart 32C and seat data 32B) in response to a user input.

[0286] When the above operation is completed, the transmission control unit 31E performs a process of transmitting the icon 263 (icon selection data 33A) to the communication device corresponding to the seat 66 (seat selection data 33B) selected by the seat selection unit 31D. After the transmission process is completed, the screen transitions to FIG. 29(e).

[0287] As shown in Fig. 29(e), an icon 263 (icon selection data 33A) is displayed on a screen 260A (display unit). In this embodiment, the seat display control unit 31C does not display the transmitted seat position 66 (seat selection data 33B) on the screen 260A (display unit) in accordance with the transmitted seat 266 (seat selection data 33B), but may display the transmitted seat position 66 (seat selection data 33B) near the icon 263 (icon selection data 33A).

[0288] As shown in FIG. 30(a), the screen 71 is mainly provided on the trim near the passengers in the rear seats, and is, for example, a screen displayed on a display unit on the interior wall of the vehicle. The screen 71 is composed of a liquid crystal panel and an electrostatic touch panel, and may have a decorative film on its surface. In this embodiment, it is an example having a wood-grain decorative film. The decorative film in this embodiment may be thinly sliced ​​wood, or may be a resin material with a wood grain printed on it. The screen 71 may also be provided on the trim near the passengers in the front seats.

[0289] As shown in FIG. 30(b), the icon display control unit 31A determines whether the user has touched the screen 71 (display unit), and if it determines that the user has touched the screen 71 (display unit), it transitions to FIG. 30(c).

[0290] As shown in FIG. 30(c), icons (72, 73, 74) (icon data 32A) are displayed in a screen 71 (display unit). In order to select one icon (icon selection data 33A) from at least one icon (72, 73, 74) (icon data 32A) in response to a user input, an icon display control unit 31A displays icons (72, 73, 74) (icon data 32A) in the screen 71 (display unit). When an icon selection unit 31B selects one icon 72 (icon selection data 33A) from at least one icon (72, 73, 74) (icon data 32A) in response to a user input, a transition is made to FIG. 30(d).

[0291] As shown in FIG. 30(d), an icon 72 (icon selection data 33A), a seating chart 75 (seating chart 32C), and seats (75A, 75B, 75C, 75D, 75E) (seat data 32B) in the seating chart 75 (seating chart 32C) are displayed in the screen 71 (display unit). In response to a user's input, the seat selection unit 31D touches with a finger a seat 75D (seat selection data 33B) that is the destination of the icon 72 (icon selection data 33A) selected by the icon selection unit 31B from the seating chart 75 (seating chart 32C). When the above operation is completed, the transmission control unit 31E performs a process of transmitting the icon 72 (icon selection data 33A) to the communication device corresponding to the seat 75D (seat selection data 33B) selected by the seat selection unit 31D. Here, the seat 75C indicates the seat of the sender.

[0292] FIG. 31 shows another embodiment of a screen transition for a user in the icon transmission process on the display unit of the communication device shown in FIG.

[0293] As shown in Fig. 31(a), the screen 81 is mainly provided on the trim near the passengers in the rear seats, and is, for example, a screen displayed on a display unit on the interior wall of the vehicle. The screen 81 is made up of a liquid crystal panel and an electrostatic touch panel, and may have a decorative film on its surface. In this embodiment, this is an example in which a wood-grain decorative film is provided. The decorative film in this embodiment may be thinly sliced ​​wood, or may be a resin material with a wood grain pattern printed on it.

[0294] As shown in FIG. 31(b), icons (82, 83, 84) (icon data 32A) are displayed in a screen 81 (display unit). In order to select one icon (icon selection data 33A) from at least one icon (82, 83, 84) (icon data 32A) in response to a user input, an icon display control unit 31A displays icons (82, 83, 84) (icon data 32A) in the screen 81 (display unit). Icons (82, 83, 84) (icon data 32A) are displayed on the screen 81 (display unit). When an icon selection unit 31B selects one icon 82 (icon selection data 33A) from at least one icon (82, 83, 84) (icon data 32A) in response to a user input, a transition to FIG. 31(c) is made.

[0295] As shown in FIG. 31(c), a seating chart 85 (seating chart 32C) is displayed on a screen 81 (display unit), and seats (85A, 85B, 85C, 85D, 85E) (seat data 32B) are displayed on the seating chart 85 (seating chart 32C). In response to a user's input, the seat selection unit 31D slides a finger from the source seat 86C (seat selection data 33B) toward the seat 85D (seat selection data 33B) that is the destination of the icon 82 (icon selection data 33A) selected by the icon selection unit 31B in the seating chart 85 (seating chart 32C). When the above operation is completed, the transmission control unit 31E performs a process of transmitting the icon 82 (icon selection data 33A) to the communication device corresponding to the seat 85D (seat selection data 33B) selected by the seat selection unit 31D.

[0296] As described above, in the vehicle according to the third embodiment, the communication control device 400 includes a seat selection unit 31D that selects, from the seat data 32B in the seat chart 32C, seat selection data 33B to which the icon selection data 33A selected by the icon selection unit 31B is to be transmitted in response to a user's input, and a transmission control unit 31E that transmits an icon to the communication device corresponding to the seat selection data 33B selected by the seat selection unit 31D. In this way, the transmission control unit 31E transmits the icon selection data 33A in response to the seat selection data 33B, thereby facilitating non-vocal communication between occupants.

[0297] In addition, screen 260A (display unit) on which communication information for the front seats is displayed is positioned away from the driver's seat, while screen 260B closer to the driver's seat displays information necessary for driving, so the driver moves his or her eyes less, allowing smooth non-audio communication with rear seat passengers without interfering with driving.

[0298] The screen (71, 81) (display unit) on which rear seat communication information is displayed is provided on the trim near the rear seat passengers, making it easy for the passengers to operate, and by selecting an icon or seat, communication is not based on text or voice, which contributes to smoother communication between passengers. In addition, the screen (71, 81) (display unit) has a decorative film on its surface, which enhances the sense of unity inside the vehicle when the screen is turned off, contributing to the design. Furthermore, when sending an icon, the passenger can slide their finger from their own seat to the seat to which they wish to send, making operation easier and contributing to ease of operation for the passengers.

[0299] (Example of reception processing of a communication control device) Next, the procedure of the icon receiving process executed by the control unit 230 will be described.

[0300] FIG. 32 is a flowchart showing an example of the procedure of an icon receiving process executed by the control unit 230 according to the present embodiment.

[0301] First, the reception control unit 31F judges whether there is information that the icon selection data 33A has been received by the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E (step S901). If it is judged that there is no received information (step S901: No), the icon selection data 33A is not received and the process of receiving the icon selection data 33A ends. If it is judged that there is received information (step S901: Yes), the process proceeds to step S902.

[0302] Next, in step S902, the seat display control unit 31C performs a process of displaying the seat positions including the transmitted seat selection data 33B and the transmitted seat chart data 33C on the display unit in response to the transmitted seat selection data 33B. The icon display control unit 31A performs a process of displaying the icon selection data 33A on the display unit in response to the transmitted icon selection data 33A. When the above process is completed, the process of receiving the icon selection data 33A executed by the control unit 230 is completed (step S902).

[0303] (Example of display format in reception processing of the display unit) Next, a screen transition for a user in the process of receiving the icon selection data 33A on the display unit of the communication device will be described with reference to FIG.

[0304] As shown in Fig. 33(a), screen 301 is a screen displayed on a display unit that is mainly displayed in front of the passengers in the front seats, and the display unit is a center display provided in the instrument panel of the vehicle, and the driver's seat is on the right side. As shown in Fig. 33(a), screen 301A, which is far from the driver's seat, displays information on icons (102, 103) related to communication, and screen 301B, which is closer to the driver's seat, displays information necessary for driving. By displaying information necessary for driving on screen 301B, the driver moves his or her eyes less, so as not to be disturbed while driving.

[0305] On the screen 301A, the seat display control unit 31C performs a process of displaying the seat position (seat selection data 33B) including the transmitted seat selection data 33B and the transmitted seat chart data 33C on the screen 301A (display unit) in response to the transmitted seat selection data 33B. The icon display control unit 31A performs a process of displaying the icon (102, 103) (icon selection data 33A) on the screen 301A (display unit) in response to the transmitted icon (102, 103) (icon selection data 33A). When the above process is completed, the process of receiving the icon selection data 33A executed by the control unit 230 is completed. Here, the icon 302 is the icon selection data 33A transmitted from seat D, and the icon 303 is the icon selection data 33A transmitted from seat C, and a plurality of pieces of the icon selection data 33A are displayed, but this is because a large number of pieces of the icon selection data 33A are transmitted, and the icon selection data 33A may be displayed overlapping each other.

[0306] As shown in FIG. 33(b), the screen 304 is mainly provided on the trim near the passengers in the rear seats, and is, for example, a screen displayed on a display unit on the inside wall of the vehicle. The screen 304 is composed of a liquid crystal panel and an electrostatic touch panel, and may have a decorative film on its surface. In this embodiment, an example is shown in which a wood-grain decorative film is provided. The decorative film in this embodiment may be thinly sliced ​​wood, or may be a resin material with a wood grain printed on it. The screen 304 may also be provided on the trim near the passengers in the front seats.

[0307] On the screen 304, icons 305 (icon selection data 33A), seating chart 306 (seating chart data 33C), and seats (106A, 106B, 106C, 106D, 106E) (seat selection data 33B) in the seating chart 306 (seating chart data 33C) are displayed. The seat display control unit 31C performs a process of displaying the seat positions (seat selection data 33B) including the transmitted seats 306C and 306D (seat selection data 33B) and the transmitted seating chart 306 (seating chart data 33C) on the screen 304 (display unit) in accordance with the transmitted icons 305 (icon selection data 33A). The icon display control unit 31A performs a process of displaying the icons 305 (icon selection data 33A) on the screen 304 (display unit) in accordance with the transmitted icons 305 (icon selection data 33A). When the above process is completed, the process executed by the control unit 230 for receiving the icon selection data 33A is completed.

[0308] Thus, according to the vehicle of the third embodiment, the communication control device 400 includes a reception control unit 31F that receives an icon to the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E, the seat display control unit 31C displays the transmitted seat position (seat selection data 33B) on the display unit according to the transmitted seat selection data 33B, and the icon display control unit 31A displays the icon selection data 33A on the display unit according to the transmitted icon selection data 33A. As a result, the reception control unit 31F displays the transmitted seat position (seat selection data 33B) and the icon selection data 33A, thereby facilitating non-audio communication between occupants.

[0309] (Example of reply processing of communication control device) Next, the procedure of the process executed by the control unit 230 for returning an icon will be described.

[0310] FIG. 34 is a flowchart showing an example of a procedure for a process of returning an icon executed by control unit 230 according to the present embodiment.

[0311] First, the icon display control unit 31A judges whether or not the user has touched the display unit (step S1101). If it is judged that the user has not touched the display unit (step S1101: No), the icon selection data 33A is not returned and the process of returning the icon selection data 33A ends. If it is judged that the user has touched the display unit (step S1101: Yes), the process proceeds to step S1102.

[0312] In step S1102, in response to a user's input, the icon display control unit 31A displays the icon data 32A in order to select one piece of icon selection data 33A from at least one piece of icon data 32A. After displaying the icon data 32A, the process proceeds to step S1103.

[0313] In step S1103, it is determined whether the user wants to change the type of icon data 32A to be transmitted. If the user determines that the type of icon data 32A is to be changed (step S1103: Yes), the process proceeds to step S1104. In step S1104, the icon display control unit 31A changes the type of icon data 32A, and the process proceeds to step S1103. If the user determines that the type of icon data 32A is not to be changed (step S1103: No), the process proceeds to step S1105.

[0314] In step S1105, icon selection unit 31B determines whether or not one piece of icon data 32A has been selected from at least one piece of icon data 32A in response to the user's input. If it is determined that the user has not selected icon data 32A (step S1105: No), the process proceeds to step S1106.

[0315] In step S1106, icon display control unit 31A determines whether a predetermined time has passed since touching the display unit. If it is determined that the predetermined time has passed (step S1106: Yes), the user does not transmit icon selection data 33A, and the process of transmitting icon selection data 33A ends. If it is determined that the predetermined time has not passed (step S1106: No), the process proceeds to step S1102. If it is determined that the user has selected icon data 32A (step S1105: Yes), the process proceeds to step S1107.

[0316] In step S1107, the transmission control unit 31E performs a process of transmitting the icon selection data 33A to the communication device corresponding to the latest source seat selection data 33B obtained by the reception control unit 31F. After performing the process of transmitting the icon selection data 33A to the communication device corresponding to the selected seat selection data 33B, the process proceeds to step S1108.

[0317] In step S1108, the light-emission control unit 31G performs a process of causing the light-emitting units arranged between the communication devices to sequentially light up toward the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E so that the light-emitting units approach from the transmitted seat selection data 33B. When the process of causing the light-emitting units to sequentially light up toward the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E so that the light-emitting units approach from the transmitted seat selection data 33B is performed, the process of returning the icon selection data 33A executed by the control unit 230 is completed.

[0318] (Example of display form in response processing of the display unit) Next, a screen transition for a user in the icon reply process on the display unit of the communication device will be described with reference to Figs.

[0319] As shown in Fig. 35(a), screen 320 is a screen displayed on a display unit that is mainly displayed in front of the passengers in the front seats, and the display unit is a center display provided in the instrument panel of the vehicle, and the driver's seat is on the right side. As shown in Fig. 35(a), screen 320A, which is far from the driver's seat, displays information on icons (121, 122) related to communication, and screen 320B, which is closer to the driver's seat, displays information necessary for driving. By displaying information necessary for driving on screen 320B, the driver moves his / her eyes less, so as not to be disturbed while driving.

[0320] The screen 320A shows a process in which the seat display control unit 31C performs a process of displaying the seat position (seat selection data 33B) including the transmitted seat selection data 33B and the transmitted seat chart data 33C on the screen 320A (display unit) in accordance with the transmitted seat selection data 33B. The screen is a screen after the icon display control unit 31A performs a process of displaying the icon (121, 122) (icon selection data 33A) on the screen 320A (display unit) in accordance with the transmitted icon (121, 122) (icon selection data 33A), that is, a screen after the icon (121, 122) (icon selection data 33A) is received. After that, when the icon display control unit 31A determines whether the user has touched the screen 320 (display unit), the screen transitions to FIG. 35(b). In this embodiment, the user touches the icon 121 of seat D.

[0321] As shown in Fig. 35(b), the screen 320A (display unit) displays icons (323, 324, 325) and seats, and the icon selection unit 31B selects one icon 323 (icon selection data 33A) from at least one icon (323, 324, 325) (icon data 32A) in response to a user input. When the above operation is completed, the transmission control unit 31E performs a process of transmitting the icon 323 (icon selection data 33A) to the communication device corresponding to the seat D (seat selection data 33B) selected by the seat selection unit 31D. After the transmission process is completed, the screen transitions to Fig. 35(c).

[0322] As shown in Fig. 35(c), an icon 323 (icon selection data 33A) is displayed on a screen 320A (display unit). In this embodiment, the seat display control unit 31C does not display the transmitted seat position D (seat selection data 33B) on the screen 320A (display unit) in accordance with the transmitted seat D (seat selection data 33B), but may display the transmitted seat position D (seat selection data 33B) near the icon 323 (icon selection data 33A).

[0323] FIG. 36 explains the screen transition when an icon is transmitted by seat C, and seat D, which has received the icon, replies to seat C. Screen 331 is mainly provided on the trim near the passengers in the rear seats, and is a screen displayed on a display unit on the inner wall of the vehicle, for example. Screen 331 is made up of a liquid crystal panel and an electrostatic touch panel, and may have a decorative film on its surface. In this embodiment, this is an example in which a wood-grain decorative film is provided. The decorative film in this embodiment may be thinly sliced ​​wood, or may be a resin material with a wood grain printed on it. Screen 331 may also be provided on the trim near the passengers in the front seats.

[0324] 36(a), an icon 332 (icon data 32A), a seating chart 333 (seating chart data 33C), and seats (333A, 333B, 333C, 333D, 333E) (seat selection data 33B) in the seating chart 333 (seating chart data 33C) are displayed on a screen 331. A seat display control unit 31C performs processing to display the seat positions including the transmitted seats 333C and 333D (seat selection data 33B) and the transmitted seating chart 333 (seating chart data 33C) on the screen 331 (display unit) in accordance with the transmitted seats 333C and 333D (seat selection data 33B). This is the screen after the icon display control unit 31A has performed processing to display the icon 332 (icon selection data 33A) on the screen 304 (display unit) in accordance with the transmitted icon 332 (icon selection data 33A), i.e., the screen after the icon 332 (icon selection data 33A) has been received.

[0325] As shown in Fig. 36(b), when the icon display control unit 31A determines whether or not the user has touched the screen 320 (display unit), the screen 331 transitions to Fig. 36(c). In this embodiment, the user has touched the screen 331.

[0326] As shown in Fig. 36(c), icons (334, 335, 336) are displayed on screen 331 (display unit), and icon selection unit 31B selects one icon 334 (icon selection data 33A) from at least one icon (334, 335, 336) (icon data 32A) in response to a user input. When the above operation is completed, transmission control unit 31E performs a process of transmitting icon 334 (icon selection data 33A) to the communication device corresponding to seat 333C (seat selection data 33B) selected by seat selection unit 31D. After the transmission process is completed, the screen transitions to Fig. 36(d).

[0327] As shown in FIG. 36(d), screen 331 (display unit) displays icon 334 (icon selection data 33A), seating chart 333 (seating chart data 33C), and within seating chart 333 (seating chart data 33C), source seat 335D and destination seat 335C (seat selection data 33B).

[0328] Thus, in the vehicle according to the third embodiment, the communication control device 400 includes a transmission control unit 31E that transmits the icon 334 (icon selection data 33A) selected by the icon selection unit 31B to the communication device corresponding to the destination seat position (seat selection data 33B) in response to a user input. In this way, the communication device that receives the icon selection data 33A transmits the icon selection data 33A in response to the seat selection data 33B, thereby facilitating non-vocal communication between occupants.

[0329] (Example of light emission form in light emission processing of light emitting part) 37 explains a method and sequence of lighting the light-emitting parts of the light-emitting device 229 by the light-emission control unit 31G of the communication control device 400 when a selected icon is transmitted corresponding to a selected seat in response to a user input. Here, the method and sequence of lighting the light-emitting parts when an occupant in seat 142C transmits icon selection data 33A to an occupant in seat 142D will be explained.

[0330] As shown in Fig. 37(a), a vehicle 141 is a schematic diagram of the vehicle seen from above, and the vehicle 141 is provided with seats (142A, 142B, 142C, 142D, 142E), communication devices (143A, 143B, 143C, 143D, 143E), and light-emitting devices (144a, 144b, 144c, 144d, 144e, 144f, 144g, 144h, 144i, 144j) between each of the communication devices (143A, 143B, 143C, 143D, 143E). Fig. 37(a) shows a state before an occupant in the seat 142C transmits icon selection data 33A to an occupant in the seat 142D.

[0331] The light-emitting units (144a, 144b, 144c, 144d, 144e, 144f, 144g, 144h, 144i, 144j) may be provided on the inner wall of the vehicle and have a decorative film on the surface. The decorative film may be thinly sliced ​​wood or a resin material with wood grain printed on it.

[0332] Next, Figure 37(b) illustrates the state of the light-emitting devices (144a, 144b, 144c, 144d, 144e, 144f, 144g, 144h, 144i, 144j) shown in Figure 37(a) after an occupant in seat 142C transmits icon selection data 33A to an occupant in seat 142D.

[0333] As shown in Fig. 37(b), the light emitting devices (144a, 144b, 144c, 144d, 144e, 144f, 144g, 144h, 144i, 144j) in Fig. 37(a) are switched to the light emitting devices (144A, 144B, 144C, 144F, 144G, 144H, 144I, 144J) as a result of the occupant of the seat 142C transmitting the icon selection data 33A to the occupant of the seat 142D, which indicates that the light emitting devices are turned on. Also, the light emitting devices are turned on in the order of the light emitting device 144C, the light emitting device 144B, the light emitting device 144A, the light emitting device 144F, the light emitting device 144G, the light emitting device 144H, the light emitting device 144I, and the light emitting device 144J.

[0334] Light-emitting devices (144A, 144B, 144C, 144d, 144e, 144F, 144G, 144H, 144I, 144J) provided between seats 142C and 142D light up in the +Y direction of the vehicle, starting from light-emitting device 144C, reaching light-emitting device 144A, lighting up along the +X direction of the vehicle from light-emitting device 144A to light-emitting device 144F, and then lighting up in the -Y direction of the vehicle, successively lighting up with light-emitting device 144J as the end point.

[0335] That is, in response to the user's input, the selected icon selection data 33A and seat selection data 33B are sequentially lit up so that the light emitting unit approaches the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E from the transmitted seat selection data 33B. The light emitting unit passes in front of the line of sight of the occupant in the traveling direction of the vehicle and flows toward the specified seat, thereby contributing to the visibility of the communication device.

[0336] Thus, according to the vehicle of the third embodiment, the display unit further includes a light-emitting unit, and the light-emitting unit includes a light-emitting control unit 31G that is arranged between each of the communication devices and sequentially lights up toward the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E as it approaches the transmitted seat selection data 33B.

[0337] As a result, the light-emitting unit sequentially lights up as it approaches the communication device corresponding to the seat selection data 33B transmitted by the transmission control unit 31E, so that not only the receiving occupant but also the occupants on board can see the icons being sent and received, which contributes to stimulating communication between occupants.

[0338] The program executed by the communication control device of this embodiment is provided in a state that it is pre-installed in a ROM or the like.

[0339] The program executed by the communication control device of this embodiment may be configured to be provided by recording it on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk) in an installable or executable format file.

[0340] Furthermore, the program executed by the communication control device of the present embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the communication control device of the present embodiment may be provided or distributed via a network such as the Internet.

[0341] In the above-described third embodiment, the communication control device 400 is mounted on a vehicle 1 having four wheels, but this is not limiting. The communication control device can be mounted on a moving object having multiple rows of seats in the front-rear direction, such as a bus, an airplane, or a train.

[0342] Regarding the above embodiment, the following A-1 to A-20 and B-1 to B-20 are disclosed.

[0343] (A-1) The first wheel; A second wheel; a vehicle body having a roof portion that is coupled to the first wheel and the second wheel, is movable in a predetermined traveling direction by the first wheel and the second wheel, covers a vehicle interior, and has an outer surface and an inner surface; a movement detection unit configured to detect movement of the vehicle body in the predetermined traveling direction; a lighting unit disposed on the inner surface of the roof unit, the illumination unit has at least a first light-emitting unit, a second light-emitting unit, and a third light-emitting unit; The first light-emitting unit, the second light-emitting unit, and the third light-emitting unit are arranged on the inner surface of the roof unit along the predetermined traveling direction, and a light-emitting state changes in response to movement of the vehicle body along the traveling direction. vehicle.

[0344] (A-2) The first light-emitting unit, the second light-emitting unit, and the third light-emitting unit change their light-emitting states in response to the movement of the vehicle body along the traveling direction and map information. The vehicle described in (A-1).

[0345] (A-3) The vehicle further includes a wireless communication unit configured to receive the map information from an external device. The vehicle described in (A-2).

[0346] (A-4) Further, an imaging unit capable of imaging the outside of the vehicle body, an imaging range of the imaging unit includes a space away from the outer surface of the roof portion based on a ground on which at least one of the first wheel and the second wheel is in contact with the ground, The first light-emitting unit, the second light-emitting unit, and the third light-emitting unit change their light-emitting states in response to the movement of the vehicle body along the traveling direction and the image captured by the imaging unit. The vehicle described in (A-1).

[0347] (A-5) The vehicle described in (A-4), wherein the first light-emitting unit, the second light-emitting unit, and the third light-emitting unit change their light-emitting states in response to the movement of the vehicle body along the direction of travel and the image captured by the imaging unit a predetermined time prior.

[0348] (A-6) The predetermined time period varies depending on the moving speed of the vehicle in a predetermined direction. The vehicle described in (A-5).

[0349] (A-7) The imaging unit is disposed at the front upper portion of the vehicle interior, A vehicle described in any one of (A-4) to (A-6).

[0350] (A-8) The vehicle according to any one of (A-1) to (A-7), wherein each of the first light-emitting unit, the second light-emitting unit, and the third light-emitting unit emits monochromatic light.

[0351] (A-9) The vehicle according to any one of (A-1) to (A-7), wherein each of the first light-emitting unit, the second light-emitting unit, and the third light-emitting unit emits light of a plurality of different colors.

[0352] (A-10) The light emission patterns of the first light emitting unit, the second light emitting unit, and the third light emitting unit are A vehicle as described in any one of (A-1) to (A-9), which changes so as to flow in a direction opposite to the traveling direction in response to movement of the vehicle body along the traveling direction.

[0353] (A-11) an acquisition unit that acquires a detection result of a movement detection unit that is configured to detect movement of a vehicle body in the predetermined traveling direction, the vehicle body having a roof portion that is connected to the first wheel and the second wheel, covers a vehicle interior, and has an outer surface and an inner surface; a control unit that controls the light emission state of at least a first light emitting unit, a second light emitting unit, and a third light emitting unit that are arranged in a row on the inner surface of the roof portion along the predetermined traveling direction; A vehicle control device comprising:

[0354] (A-12) The vehicle control device described in (A-11), wherein the control unit changes the light-emitting state of the first light-emitting unit, the second light-emitting unit, and the third light-emitting unit in response to movement of the vehicle body along the direction of travel and map information.

[0355] (A-13) The vehicle control device described in (A-12), wherein the acquisition unit acquires the map information received from an external device via a wireless communication unit.

[0356] (A-14) the acquisition unit acquires an image captured by an imaging unit capable of capturing an image of the outside of the vehicle body and having an imaging range including a space away from the outer surface of the roof portion based on a ground on which at least one of the first wheel and the second wheel is in contact with the ground, The vehicle control device described in (A-11), wherein the control unit changes the light emitting state of the first light emitting unit, the second light emitting unit, and the third light emitting unit in response to the movement of the vehicle body along the direction of travel and the image captured by the imaging unit.

[0357] (A-15) The vehicle control device described in (A-14), wherein the control unit changes the light emitting state of the first light emitting unit, the second light emitting unit, and the third light emitting unit in response to the movement of the vehicle body along the direction of travel and an image captured by the imaging unit a predetermined time ago.

[0358] (A-16) The vehicle control device described in (A-15), wherein the control unit changes the specified time depending on the moving speed of the vehicle in a specified direction.

[0359] (A-17) The vehicle control device according to any one of (A-14) to (A-16), wherein the imaging unit is disposed at an upper front portion of the vehicle interior.

[0360] (A-18) The vehicle control device according to any one of (A-11) to (A-17), wherein the control unit causes each of the first light-emitting unit, the second light-emitting unit, and the third light-emitting unit to emit monochromatic light.

[0361] (A-19) A vehicle control device described in any one of (A-11) to (A-17), wherein the control unit causes each of the first light-emitting unit, the second light-emitting unit, and the third light-emitting unit to emit light of multiple colors.

[0362] (A-20) A vehicle control device described in any one of (A-11) to (A-19), wherein the control unit changes the light emission patterns of the first light-emitting unit, the second light-emitting unit, and the third light-emitting unit so that they flow in a direction opposite to the traveling direction in response to movement of the vehicle body along the traveling direction.

[0363] (B-1) A first wheel; A second wheel; a vehicle body coupled to the first wheel and the second wheel and movable by the first wheel and the second wheel; A plurality of passenger seats disposed in the vehicle body; A vehicle equipped with a communication device used for communication between the occupants, an icon display control unit that controls displaying at least one icon on a display unit, the icon display control unit being used for communication with the occupant; an icon selection unit that selects one icon from the at least one icon in response to an input from a user; a seat display control unit that controls displaying a seat chart indicating the positions of the seats on the display unit; a seat selection unit that selects, in response to an input from the user, a seat to which the icon selected by the icon selection unit is to be transmitted, from the seat chart; a transmission control unit that transmits the icon to the communication device corresponding to the seat selected by the seat selection unit.

[0364] (B-2) a reception control unit that receives the icon transmitted by the transmission control unit to the communication device corresponding to the seat, The seat display control unit displays the transmitted seat position on the display unit according to the transmitted seat position, The vehicle according to (B-1), wherein the icon display control unit displays the icon on a display unit in response to the transmitted icon.

[0365] (B-3) the communication device is disposed in each of the plurality of seats, The vehicle according to (B-1) or (B-2), wherein the display unit is provided on an inner wall of the vehicle.

[0366] (B-4) The vehicle according to any one of (B-1) to (B-3), wherein the display unit is composed of at least a liquid crystal panel and an electrostatic touch panel.

[0367] (B-5) The vehicle according to any one of (B-1) to (B-4), wherein the display unit further includes a decorative film on a surface layer.

[0368] (B-6) The vehicle according to (B-5), wherein the decorative film has a wood grain.

[0369] (B-7) the communication device is disposed between the front of each front seat; The vehicle according to (B-1) or (B-2), wherein the display unit is provided on an instrument panel of the vehicle.

[0370] (B-8) The vehicle described in (B-7), wherein the display unit is positioned at a position far from the driver's seat.

[0371] (B-9) The display unit further includes a light emitting unit, the light emitting unit is disposed between each of the communication devices, The vehicle described in (B-3) is provided with a light-emitting control unit that sequentially lights up the light-emitting unit toward the communication device corresponding to the seat transmitted by the transmission control unit as the light approaches from the seat that transmitted the signal.

[0372] (B-10) The light emitting unit is provided on an inner wall of the vehicle, The vehicle described in (B-9), wherein the light-emitting portion has a decorative film on a surface thereof.

[0373] (B-11) A first wheel; A second wheel; a vehicle body coupled to the first wheel and the second wheel and movable by the first wheel and the second wheel; A plurality of passenger seats disposed in the vehicle body; A communication control device that can be mounted in a vehicle and includes a communication device disposed in each of the plurality of seats, an icon display control unit that controls displaying at least one icon on a display unit, the icon display control unit being used for communication with the occupant; an icon selection unit that selects one icon from the at least one icon in response to an input from a user; a seat display control unit that controls displaying a seat chart indicating the positions of the seats on the display unit; a seat selection unit that selects, in response to an input from the user, a seat to which the icon selected by the icon selection unit is to be transmitted, from the seat chart; a transmission control unit that transmits the icon to the communication device corresponding to the seat selected by the seat selection unit.

[0374] (B-12) a reception control unit that receives the icon transmitted by the transmission control unit to the communication device corresponding to the seat, The seat display control unit displays the transmitted seat position on the display unit according to the transmitted seat position, The communication control device according to (B-11), wherein the icon display control unit displays the icon on a display unit in response to the transmitted icon.

[0375] (B-13) the communication device is disposed in each of the plurality of seats, The communication control device according to (B-11) or (B-12), wherein the display unit is provided on an inner wall of the vehicle.

[0376] (B-14) The communication control device according to any one of (B-11) to (B-13), wherein the display unit is composed of at least a liquid crystal panel and an electrostatic touch panel.

[0377] (B-15) The communication control device according to any one of (B-11) to (B-14), wherein the display unit further includes a decorative film on a surface thereof.

[0378] (B-16) The communication control device described in (B-15), wherein the decorative film has a wood grain.

[0379] (B-17) the communication device is disposed between each of the front seats; The communication control device according to (B-11) or (B-12), wherein the display unit is provided on an inner wall of the vehicle.

[0380] (B-18) The communication control device described in (B-17), wherein the display unit is positioned at a position far from the driver's seat.

[0381] (B-19) The display unit further includes a light emitting unit, the light emitting unit is disposed between each of the communication devices, A communication control device as described in (B-13), which is provided with an illumination control unit that sequentially lights up the illumination unit toward the communication device corresponding to the seat transmitted by the transmission control unit as the illumination unit approaches from the seat that transmitted the illumination.

[0382] (B-20) The light emitting unit is provided on an inner wall of the vehicle, The communication control device described in (B-19), wherein the light-emitting unit has a decorative film on its surface.

[0383] Although the embodiments of the present disclosure have been described above, the above-mentioned embodiments are presented as examples and are not intended to limit the scope of the invention. These new embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These new embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. Furthermore, components across different embodiments and modifications may be appropriately combined.

[0384] Furthermore, the effects of the embodiments described in this specification are merely examples and are not limiting, and other effects may also be obtained. [Explanation of symbols]

[0385] 1 vehicle 2. Body 3 wheels 6. Vehicle exterior imaging unit 7. In-vehicle imaging unit 20 Vehicle control device

Claims

1. A first wheel; A second wheel; a vehicle body coupled to the first wheel and the second wheel and movable by the first wheel and the second wheel; a first imaging unit that captures an image of the outside of the vehicle; A vehicle including a second imaging unit that images an interior of the vehicle, setting a monitoring target based on an image captured by the second imaging unit and a past image captured by the second imaging unit; When a monitoring start condition is met, a monitoring process is started to determine whether or not the monitoring target belongs to a monitoring range based on an image captured by the first imaging unit, and when a monitoring end condition is met, the monitoring process is ended. A vehicle, wherein the monitoring start condition or the monitoring end condition includes a condition related to operation of the vehicle.

2. 2. A vehicle as claimed in claim 1, The monitoring start condition includes a parking brake of the vehicle being applied.

3. 3. A vehicle according to claim 2, A vehicle, wherein the monitoring start condition includes the vehicle being unlocked from the outside.

4. A vehicle according to claim 2 or 3, A vehicle, wherein the monitoring start condition includes a door of the vehicle being opened.

5. A vehicle according to any one of claims 2 to 4, The monitoring start condition includes extraction of the monitoring target based on an image captured by the first imaging unit.

6. A vehicle according to any one of claims 1 to 5, The monitoring termination condition includes a parking brake of the vehicle being released.

7. A vehicle according to any one of claims 1 to 6, A vehicle, wherein the monitoring end condition includes the vehicle being locked from the outside.

8. A vehicle according to any one of claims 1 to 7, In the monitoring process, if it is determined that the monitoring target does not belong to a monitoring range based on an image captured by the first imaging unit, a notification process is performed.

9. A vehicle according to any one of claims 1 to 8, A vehicle, wherein the monitoring range is based on an imaging range of the first imaging unit.

10. A first wheel; A second wheel; a vehicle body coupled to the first wheel and the second wheel and movable by the first wheel and the second wheel; a first imaging unit that captures an image of the outside of the vehicle; A vehicle control device that can be mounted on a vehicle including a second imaging unit that images an interior of the vehicle, setting a monitoring target based on an image captured by the second imaging unit and a past image captured by the second imaging unit; When a monitoring start condition is met, a monitoring process is started to determine whether or not the monitoring target belongs to a monitoring range based on an image captured by the first imaging unit, and when a monitoring end condition is met, the monitoring process is ended. A vehicle control device, wherein the monitoring start condition or the monitoring end condition includes a condition related to operation of the vehicle.

11. The vehicle control device according to claim 10, The monitoring start condition includes a parking brake of the vehicle being applied.

12. The vehicle control device according to claim 11, The monitoring start condition includes a state in which the vehicle is unlocked from the outside.

13. The vehicle control device according to claim 11 or 12, The vehicle control device, wherein the monitoring start condition includes a door of the vehicle being opened.

14. A vehicle control device according to any one of claims 11 to 13, The vehicle control device, wherein the monitoring start condition includes extraction of the monitoring target based on an image captured by the first imaging unit.

15. A vehicle control device according to any one of claims 10 to 14, The monitoring end condition includes that a parking brake of the vehicle is released.

16. A vehicle control device according to any one of claims 10 to 15, A vehicle control device, wherein the monitoring end condition includes a condition in which the vehicle is locked from the outside.

17. A vehicle control device according to any one of claims 10 to 16, In the monitoring process, when it is determined that the monitoring target does not belong to a monitoring range based on the image captured by the first imaging unit, a notification process is performed.

18. A vehicle control device according to any one of claims 10 to 17, A vehicle control device, wherein the monitoring range is based on an imaging range of the first imaging unit.

Images