Vehicle control device and vehicle control method

Abstract

A vehicle control device (11) determines whether or not luminosity of a vehicle cabin of a vehicle is sufficient, controls an illumination device (15) so as to illuminate the inside of the vehicle cabin with illumination light when determination is made that the luminosity of the vehicle cabin of the vehicle is not sufficient, and transmits, externally from the vehicle, an image of the vehicle cabin that is shot by a camera (16) after controlling the illumination device (15).

Description

Vehicle control device and vehicle control method

[0001] The present invention relates to a vehicle control device and a vehicle control method.

[0002] A technology is known in which, when the door mirror is operated and the camera is ready to photograph the interior of the vehicle, the camera performs a vehicle photography process to photograph the interior of the vehicle, and transmits the photographed image of the interior of the vehicle (Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-46148

[0004] However, the technique described in Patent Document 1 has a problem in that when the interior of the vehicle is dark, the captured image becomes unclear.

[0005] The problem to be solved by the present invention is to provide a vehicle control device and a vehicle control method that can provide a clear image of the interior of a vehicle even when the interior of the vehicle is dark.

[0006] The present invention solves the above problem by controlling a lighting device to irradiate illumination light into the vehicle interior when it is determined that the brightness inside the vehicle's interior is insufficient, and then transmitting an image of the interior of the vehicle interior taken by a camera to the outside of the vehicle after controlling the lighting device.

[0007] According to the present invention, even when the interior of a vehicle is dark, a clear image of the interior of the vehicle can be provided.

[0008] FIG. 1 is a diagram showing a block configuration of a vehicle control system including a vehicle control device according to an embodiment of the present invention. FIG. 2 is a diagram for explaining a method for determining illuminance according to the embodiment. FIG. 3 is a diagram showing an example of lighting control according to the embodiment. FIG. 4 is a diagram showing an example of lighting control according to the embodiment. FIG. 5 is a sequence chart for explaining a vehicle control method executed by the vehicle control system according to the embodiment. FIG. 6 is a sequence chart for explaining a vehicle control method executed by the vehicle control system according to the embodiment. FIG. 7 is a sequence chart for explaining a vehicle control method executed by the vehicle control system according to the embodiment.

[0009] First Embodiment Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] FIG. 1 is a block diagram showing a vehicle control system including a vehicle control device according to an embodiment of the present invention. The vehicle control system 1 is a system for controlling various devices provided in a vehicle to capture images of the interior of the vehicle and transmit the captured images to an external device of the vehicle. As shown in FIG. 1 , the vehicle control system 1 includes an in-vehicle system 10 and a terminal device 20 mounted in the vehicle. The terminal device 20 is a device used by a user, such as a vehicle occupant. The in-vehicle system 10 and the terminal device 20 can exchange information with each other via a network. The network refers to a telecommunications network such as the Internet or a LAN. Any known network can be used as long as it allows information to be exchanged between the in-vehicle system 10 and the terminal device 20. In this embodiment, when the user of the terminal device 20 gets off the vehicle and leaves the vehicle, for example, and the terminal device 20 is located away from the vehicle, the in-vehicle system 10 and the terminal device 20 exchange information with each other via the network.

[0011] 1, the in-vehicle system 10 includes a vehicle control device 11, an in-vehicle communication device 12, a storage 13, an illuminance acquisition device 14, a lighting device 15, and a camera 16. The devices included in the in-vehicle system 10 are connected to each other via a Controller Area Network (CAN) or other in-vehicle LAN for transmitting and receiving information to and from each other. Each device will be described below.

[0012] The vehicle control device 11 includes a processor 100. The processor 100 is a computer having hardware and software, and includes a ROM storing a program, a CPU that executes the program stored in the ROM, and a RAM that functions as an accessible storage device. The processor 100 includes functional blocks, such as an illuminance determination unit 101, an illumination control unit 102, an image capture control unit 103, and a transmission unit 104, and executes each function through cooperation between the software and hardware for realizing each function or executing each process. Note that in this embodiment, the functions of the processor 100 are divided into four blocks, and the functions of each functional block are described. However, the functions of the processor 100 do not necessarily have to be divided into four blocks, and may be divided into three or fewer functional blocks, or five or more functional blocks.

[0013] The illuminance determination unit 101 executes an illuminance determination process to determine whether the brightness inside the vehicle cabin is sufficient. Sufficient brightness means that there is sufficient brightness to capture an image of the interior of the vehicle cabin. In the illuminance determination process, the illuminance determination unit 101 determines whether the brightness inside the vehicle cabin is sufficient based on an image of the interior of the vehicle cabin. Specifically, the illuminance determination unit 101 acquires information on the illuminance inside the vehicle cabin acquired from the image of the interior of the vehicle cabin from the illuminance acquisition device 14 and determines whether the illuminance is equal to or greater than a predetermined threshold. The predetermined threshold is a value that indicates the brightness required to capture an image of the interior of the vehicle cabin and is acquired through experiments or simulations. If the illuminance is equal to or greater than the predetermined threshold, the illuminance determination unit 101 determines that the brightness inside the vehicle cabin is sufficient. If the illuminance is less than the predetermined threshold, the illuminance determination unit 101 determines that the brightness inside the vehicle cabin is insufficient. Note that in this embodiment, the brightness inside the vehicle cabin is not limited to being determined based on illuminance information acquired from the image of the interior of the vehicle cabin, and the brightness inside the vehicle cabin may also be determined based on luminance information acquired from the image of the interior of the vehicle cabin.

[0014] Furthermore, as long as the brightness is determined based on the illuminance, it is not limited to determining whether the illuminance is equal to or greater than a predetermined threshold, and other methods may be used. Here, an example of an illuminance determination method will be described with reference to FIG. 2. FIG. 2 is a diagram for explaining the illuminance determination method according to this embodiment. FIG. 2 is a diagram showing an example of a histogram of an image captured inside the vehicle cabin. The histogram shows a density distribution related to illuminance. As an example of the brightness determination method, the illuminance determination unit 101 may determine that the brightness inside the vehicle cabin is sufficient when the distribution of illuminances equal to or greater than the target illuminance is a predetermined percentage or more, and may determine that the brightness inside the vehicle cabin is insufficient when the distribution of illuminances equal to or greater than the target illuminance is less than the predetermined percentage.

[0015] For example, the illuminance determination unit 101 may determine that the brightness in the vehicle interior is sufficient when the average illuminance is equal to or greater than a predetermined threshold, and may determine that the brightness in the vehicle interior is insufficient when the average illuminance is less than the predetermined threshold. Note that other statistical quantities may be used instead of the average illuminance. The illuminance determination unit 101 may determine that the brightness in the vehicle interior is sufficient when all of the top peak values ​​in the illuminance distribution are equal to or greater than the target illuminance, and may determine that the brightness in the vehicle interior is insufficient when all of the top peak values ​​in the illuminance distribution are not equal to or greater than the target illuminance. The number of top peak values ​​may be one or more. In the example of FIG. 2 , the illuminances at Peaks 1 and 2 are equal to or greater than the target illuminance, and the illuminances at Peaks 3 and 4 are less than the target illuminance. The illuminance determination unit 101 determines that the brightness in the vehicle interior is insufficient because all of the top peak values ​​in the illuminance distribution are not equal to or greater than the target illuminance.

[0016] In this embodiment, the illuminance determination process is executed when an image acquisition instruction to acquire an image of the vehicle's interior is received. The image acquisition instruction is issued by a user via the terminal device 20. For example, when the illuminance determination unit 101 receives the image acquisition instruction from the terminal device 20 via the in-vehicle communication device 12, it activates the camera of the illuminance acquisition device 14 and determines whether the brightness of the vehicle interior is sufficient based on an image of the vehicle interior captured when the camera is activated. The illuminance determination process is also executed when an occupant exits the vehicle. For example, the illuminance determination unit 101 activates the camera of the illuminance acquisition device 14 when an occupant exits the vehicle and acquires an image of the vehicle interior captured by the camera. The illuminance determination unit 101 determines whether the brightness of the vehicle interior is sufficient based on the acquired image of the vehicle interior. Note that the illuminance acquisition device 14 and the interior cameras such as the camera 16 may continue to operate while the vehicle ignition is on and acquire images of the vehicle interior at regular intervals.

[0017] In this embodiment, the illuminance determination unit 101 determines whether the brightness in the vehicle interior is sufficient. However, the determination target may be the brightness of the entire vehicle interior or the brightness of a portion of the vehicle interior. For example, when the brightness of the entire vehicle interior is determined, an image of the vehicle interior that is zoomed out so that the entire vehicle interior is included in the angle of view is used. When the brightness of a portion of the vehicle interior is determined, an image of the vehicle interior that is zoomed in to a portion of the vehicle interior is used. In this embodiment, as will be described later, the brightness of the region S in the vehicle interior is determined based on an image of the vehicle interior captured in an area where the display angle of view and the illumination region of the lighting device 15 overlap (region S).

[0018] The lighting control unit 102 executes a lighting control process to control the lighting devices 15. In the lighting control process, the lighting control unit 102 controls the lighting devices 15 to irradiate illumination light into the vehicle interior. Specifically, when it is determined that the brightness in the vehicle interior is insufficient, the lighting control unit 102 outputs a control instruction to the lighting devices 15 to turn them on. When it is determined that the brightness in the vehicle interior is sufficient, the lighting control unit 102 outputs a control instruction to the lighting devices 15 to maintain the lighting state or to turn them off. The lighting devices 15 to be controlled may be all or some of the lighting devices 15 in the vehicle interior.

[0019] Here, an example will be described in which some of the lighting devices are controlled. For example, the lighting control unit 102 may control lighting devices 15, among the multiple lighting devices 15 in the vehicle cabin, that can irradiate a predetermined shooting area in the vehicle cabin with illumination light. The predetermined shooting area is an area specified by a user. In this embodiment, the shooting area is expressed by an angle of view. The lighting devices 15 that can irradiate the predetermined shooting area with illumination light are lighting devices whose irradiation area that can irradiate illumination light is superimposed on the shooting area. The illumination area of ​​each lighting device 15 is determined in advance and stored in the storage 13. In this embodiment, a shooting area instruction that specifies the shooting area in the vehicle cabin is included in the image acquisition instruction. When the lighting control unit 102 receives an image acquisition instruction that includes a shooting area instruction, it acquires the illumination area of ​​each lighting device 15 from the storage 13 and compares the illumination area of ​​each of the multiple lighting devices 15 with the specified shooting area. The lighting control unit 102 identifies, from among the plurality of lighting devices 15, a lighting device 15 whose lighting area overlaps with the shooting area, and controls the identified lighting device 15 to irradiate illumination light into the vehicle interior.

[0020] An example of lighting control will now be described with reference to FIG. 3 . FIG. 3 is a diagram illustrating an example of lighting control according to this embodiment. The diagram on the right side of FIG. 3 is an overall diagram illustrating the lighting areas and display angles of view of all lighting devices 15, and each diagram on the left side of FIG. 3 is a diagram illustrating the positional relationship between the lighting areas and display angles of view of each lighting device 15. In this example, four lighting devices 151, 152, 153, and 154 are installed as the lighting devices 15. In this embodiment, the display angle of view of the camera 16 is expressed by angles θ and φ relative to the mounting point of the camera 16. The angle θ is defined as the angle around the x-axis (the axis in the longitudinal direction of the vehicle) relative to the mounting point of the camera 16. The angle φ is defined as the angle around the y-axis (the axis in the transverse direction of the vehicle) relative to the mounting point of the camera 16. In FIG. 3 , the horizontal axis represents the angle θ around the x-axis, and the vertical axis represents the angle φ around the y-axis.

[0021] In FIG. 3 , area NR indicates an area that cannot be photographed by the camera 16. Areas other than area NR are areas that can be photographed by controlling the display angle of view of the camera 16. Areas L1, L2, L3, and L4 indicate illumination areas that can be irradiated with illumination light by the lighting devices 151, 152, 153, and 154, respectively. Each illumination area may or may not have overlapping areas. Area BR indicates an area that does not require illumination, which is sufficiently bright even without illumination light from the lighting device 15. The illumination area and the area that does not require illumination are set in advance and stored in the storage 13. Area DR is the display angle of view of the camera 16, and is defined by each vertex of a quadrangle. Each vertex of area DR is defined by (θ １ , φ １ ), (θ ２ , φ ２ ), (θ3, φ3), (θ ４ , φ ４ ) is represented as

[0022] In this embodiment, the illuminance determination unit 101 calculates an area (area S) where the display angle of view DR and the illumination areas of each illumination device 15 overlap, and determines the brightness based on the illuminance and / or luminance within the area S. The illumination control unit 102 controls the lighting devices 15 to be turned on and off in accordance with the brightness determination result. As shown in the left diagram of FIG. 3 , the entire area S where the illumination area L1 of the illumination device 151 and the display angle of view DR overlap is included in the area BR. Therefore, the illumination control unit 102 controls the illumination device 151 to be turned off. Furthermore, in the area S where the illumination area L2 of the illumination device 152 and the display angle of view DR overlap, there is an area that does not overlap with the area BR. Therefore, the illumination control unit 102 controls the illumination device 152 to be turned on. Furthermore, since there is no region S where the illumination region L3 of the illumination device 153 overlaps with the display angle of view DR, the illumination control unit 102 controls the illumination device 153 to turn off the illumination device 153. Furthermore, the region S where the illumination region L4 of the illumination device 154 overlaps with the display angle of view DR includes a region that does not overlap with the region BR. Therefore, the illumination control unit 102 controls the illumination device 154 to turn on the illumination device 154.

[0023] In this embodiment, the lighting device may be controlled not only based on the positional relationship between the area S and the area BR but also based on the illuminance of the area S. For example, the illuminance determination unit 101 determines whether the illuminance in the area S is equal to or greater than a threshold. If it is determined that the illuminance in the area S is equal to or greater than the threshold, the lighting control unit 102 controls the lighting device 15 to turn off the lighting device 15. If it is determined that the illuminance in the area S is less than the threshold, the lighting control unit 102 controls the lighting device 15 to turn on the lighting device 15.

[0024] The predetermined photographing area may also be an area where a left-behind item is detected. When a left-behind item is detected in the vehicle cabin, the lighting control unit 102 compares the lighting area of ​​each of the plurality of lighting devices 15 with the position of the left-behind item, and identifies the lighting device 15 within whose lighting area the left-behind item is located. The lighting control unit 102 controls the identified lighting device 15 to irradiate illumination light into the vehicle cabin.

[0025] Here, an example of lighting control when an item is detected left behind in the vehicle cabin will be described with reference to FIG. 4 . FIG. 4 is a diagram illustrating an example of lighting control according to this embodiment. The diagram on the right side of FIG. 4 is an overall diagram showing the illumination areas and display angles of view of all lighting devices 15, and each diagram on the left side of FIG. 4 is a diagram for explaining the positional relationship between the illumination area and display angle of view of each lighting device 15. FIG. 4 is a modified example of FIG. 3 , and since areas NR, L1, L2, L3, L4, and DR are the same as those in FIG. 3 , their description will be omitted and the description of FIG. 3 will be used as appropriate. Area LR indicates the angle of view where the left-behind item was detected. In this embodiment, the illuminance determination unit 101 calculates an area (area T) where area LR and the illumination areas of each lighting device 15 overlap, and controls the turning on and off of the lighting device 15 based on area T.

[0026] As shown in the left diagram of FIG. 4 , there is no region T where the illumination region L1 of illumination device 151 overlaps with region LR. Therefore, the illumination control unit 102 controls illumination device 151 to turn it off. Furthermore, there is a region T where the illumination region L2 of illumination device 152 overlaps with region LR. Therefore, the illumination control unit 102 controls illumination device 152 to turn it on. Furthermore, there is no region T where the illumination region L3 of illumination device 153 overlaps with region LR. Therefore, the illumination control unit 102 controls illumination device 153 to turn it off. Furthermore, there is no region T where the illumination region L4 of illumination device 154 overlaps with region LR. Therefore, the illumination control unit 102 controls illumination device 154 to turn it off. In this embodiment, the lighting control unit 102 controls the lighting device 15 so that it turns on even when it is bright, in order to make left-behind items more noticeable. However, this is not limited to this, and the lighting device 15 may be turned on only when it is dark, in combination with control of the lighting device 15 according to the display angle of view.

[0027] The photography control unit 103 executes photography control processing to acquire images of the interior of the vehicle using the camera 16. In the photography control processing, the photography control unit 103 activates the camera 16 and then periodically acquires images of the interior of the vehicle from the camera 16. Specifically, the photography control unit 103 outputs a photography instruction to the camera 16 to capture images of the interior of the vehicle. For example, the photography control processing is executed after the lighting control unit 102 controls the lighting device 15 to be turned on.

[0028] Furthermore, in the present embodiment, when an image acquisition instruction is received, the imaging control unit 103 may activate the camera 16. The imaging control unit 103 acquires an image of the interior of the vehicle cabin using the activated camera 16. Furthermore, when the image acquisition instruction includes an imaging area instruction, the imaging control unit 103 controls the angle of view of the camera 16 so as to capture a predetermined imaging area in accordance with the imaging area instruction. Specifically, the imaging control unit 103 outputs an imaging instruction to the camera 16 to capture an image of the imaging area indicated in the imaging area instruction. Note that, in the present embodiment, at the time of activation, the camera 16 may capture an image of the interior of the vehicle cabin in a zoomed-out state so that the entire interior of the vehicle cabin is included in the angle of view.

[0029] The photography control unit 103 may also detect any lost items in the vehicle interior based on an image of the vehicle interior acquired when the camera 16 is activated. For example, an image of the vehicle interior captured after the lighting device 15 is turned on is used to detect any lost items. The photography control unit 103 compares the image of the vehicle interior acquired when the camera 16 is activated with an image of the vehicle interior in a state where there is no lost item, and detects that there is a lost item if there is a difference between the acquired image of the vehicle interior in a state where there is no lost item. The image of the vehicle interior in a state where there is no lost item is an image captured in advance of the vehicle interior in a state where there is no lost item, and is stored in the storage 13. The photography control unit 103 may also capture an image zoomed in on the lost item when detecting a lost item. In addition to the camera 16, a camera for lost item detection may also be used to detect lost items.

[0030] The transmitter 104 executes a transmission process to transmit an image of the interior of the vehicle to the outside of the vehicle, which is, for example, the terminal device 20. In the transmission process, the transmitter 104 transmits the acquired image of the interior of the vehicle to the terminal communication device 24 of the terminal device 20 via the in-vehicle communication device 12. In this embodiment, the transmitter 104 transmits to the terminal device 20 an image of the interior of the vehicle that was captured after the lighting device 15 was controlled to be turned on.

[0031] Furthermore, if an item is left behind in the vehicle interior, the transmitter 104 may transmit an image of the vehicle interior taken after controlling the lighting device 15, and notification information indicating that the item is left behind in the vehicle interior. The notification information includes a highlighting display that highlights the item by, for example, surrounding it with a frame.

[0032] The in-vehicle communication device 12 is a communication interface compatible with various communication standards such as a wired LAN standard, a wireless LAN standard, etc. The in-vehicle communication device 12 is a device for transmitting and receiving information via a network between the in-vehicle system 10 and the terminal device 20, and is not particularly limited as long as it is a device capable of communicating with other devices via a network such as the Internet, and any known device can be used.

[0033] The storage 13 is a storage medium that stores various information in advance. The stored information includes information about the illumination areas of each lighting device 15. Furthermore, the storage 13 may also store areas that do not require illumination, which are sufficiently bright even without illumination from the lighting device 15. The information about these areas is defined as 3D data that represents a three-dimensional space including the three-dimensional shape of the seats in the vehicle's interior. Furthermore, the storage 13 may store an image of the interior of the vehicle with no items left behind.

[0034] The illuminance acquisition device 14 is a sensor that acquires information about the illuminance inside the vehicle cabin. For example, the illuminance acquisition device 14 includes a camera. The illuminance acquisition device 14 acquires the illuminance information from an image of the inside of the vehicle cabin captured by the camera. The acquired illuminance information is output to the vehicle control device 11. Note that the illuminance acquisition device 14 may also serve as the camera 16.

[0035] The lighting device 15 is a device that irradiates illumination light into the vehicle interior. One or more lighting devices 15 are installed in the vehicle interior. The lighting device 15 is, for example, a room lamp. The lighting devices 15 can be individually controlled, and turn on or off when a control instruction to turn on or off is output from the vehicle control device 11. For each lighting device 15, an area that can be irradiated with illumination light is determined as an illumination area, and this is stored in the storage 13.

[0036] The camera 16 is a camera using an imaging element such as a CCD, and captures images of the interior of the vehicle. The number and location of the cameras 16 are not particularly limited; for example, one camera is installed near the ceiling of the vehicle interior. Images of the interior of the vehicle captured by the camera 16 are output to the vehicle control device 11. In this embodiment, when the camera 16 receives a capture instruction to capture the interior of the vehicle from the vehicle control device 11, the camera 16 captures images of the interior of the vehicle at regular intervals and outputs the captured images of the interior of the vehicle to the vehicle control device 11. The camera 16 also controls the angle of view by zooming in to get closer to the subject or zooming out to get farther away from the subject. The camera 16 also controls the angle of view by changing the angle around the x-axis (the axis extending in the longitudinal direction of the vehicle) and the angle around the y-axis (the axis extending in the transverse direction of the vehicle). For example, when the camera 16 receives a capture area instruction, the camera 16 controls the angle of view to capture the capture area specified in the capture area instruction.

[0037] The terminal device 20 is a device operated by a user, and is, for example, a personal computer, a smartphone, a PDA, or other portable device. The user is, for example, a vehicle user. After getting out of the vehicle, the user can check an image of the interior of the vehicle via an application on the terminal device 20. The terminal device 20 includes a terminal control device 21, an output device 22, an input device 23, and a terminal communication device 24. The devices included in the terminal device 20 are connected via a wired or wireless LAN, etc., and can exchange information with each other.

[0038] The terminal control device 21 is a device that controls and cooperates with devices included in the terminal device 20 to send image acquisition instructions to the vehicle control device 11 and display images of the interior of the vehicle transmitted from the vehicle control device 11. The terminal control device 21 includes a processor for realizing these functions. The processor includes a ROM that stores a program, a CPU that is an operating circuit that functions as the terminal control device 21 by executing the program stored in the ROM, and a RAM that functions as an accessible storage device.

[0039] The terminal control device 21 outputs information to the output device 22. The output information includes, for example, an image of the interior of the vehicle cabin. When the terminal control device 21 acquires an image of the interior of the vehicle cabin from the vehicle control device 11 via the terminal communication device 24, the terminal control device 21 outputs the image of the interior of the vehicle cabin to the output device 22. When notification information indicating that an item has been left behind in the vehicle cabin is transmitted from the vehicle control device 11, the terminal control device 21 outputs the notification information to the output device 22.

[0040] For example, when a user presses a start switch for an application for displaying an image of the interior of a vehicle, the terminal control device 21 starts the application. After starting the application, the terminal control device 21 displays an application screen including an image of the interior of the vehicle on the output device 22. Note that in this embodiment, the terminal control device 21 is not limited to displaying an image of the interior of the vehicle when the application is started, and may also display an input screen for an instruction to obtain an image when the application is started, as described below.

[0041] The terminal control device 21 acquires an image acquisition instruction to acquire an image of the interior of the vehicle. For example, when a user inputs an image acquisition instruction to the input device 23, the terminal control device 21 acquires the image acquisition instruction from the input device 23. The terminal control device 21 transmits the image acquisition instruction to the vehicle control device 11 via the terminal communication device 24. The image acquisition instruction is input via an input screen for image acquisition instructions that is displayed when the application is started. In addition, a shooting area instruction may be acquired together with the image acquisition instruction.

[0042] The terminal communication device 24 is a communication interface compatible with various communication standards such as wireless LAN standards. The terminal communication device 24 is a device for transmitting and receiving information via a network between the vehicle control device 11 and the terminal device 20, and is not particularly limited as long as it is a device capable of communicating with other devices via a network such as the Internet, and any known device can be used.

[0043] The output device 22 is a device for outputting information to the user. The output device 22 is a device for displaying images, and an example thereof is a liquid crystal display. When a control instruction to display an application screen is input from the terminal control device 21, the output device 22 displays the application screen including an image of the interior of the vehicle. The output device 22 may include an operation screen for the camera 16 on the application screen. The output device 22 may display notification information indicating that an item has been left behind. The output device 22 may also be equipped with a speaker or the like to output audio information.

[0044] The input device 23 is a device through which the user inputs operations to the terminal device 20, and is, for example, a touch panel that accepts input by the user's finger or the like. The touch panel that is the input device 23 may also be provided on the liquid crystal display that is the output device 22. For example, the input device 23 accepts operation inputs by the user for image acquisition instructions and shooting area instructions on an operation screen of an application screen. This allows the user to request images of the interior of the vehicle cabin via the application of the terminal device 20, or to request images of the interior of the vehicle cabin including a specified shooting area by operating the pan, zoom, etc. of the camera 16.

[0045] Next, the procedure of the vehicle control method in the vehicle control system of this embodiment will be described with reference to the sequence chart of Fig. 5. Fig. 5 is a sequence chart for explaining the vehicle control method executed by the vehicle control system of this embodiment. When the user presses a start switch of an application for displaying an image of the interior of the vehicle on the terminal device 20, the terminal device 20 starts the control process from step S10.

[0046] In step S10, the terminal device 20 launches an application. In step S11, the terminal device 20 transmits an image acquisition instruction to the in-vehicle system 10. The image acquisition instruction may be transmitted to the in-vehicle system 10 by launching an application, or may be transmitted to the in-vehicle system 10 when a user inputs an image acquisition instruction via the application. In step S12, the in-vehicle system 10 receives the image acquisition instruction from the terminal device 20. In step S13, the in-vehicle system 10 launches the camera in the vehicle cabin (the camera of the illuminance acquisition device 14 and / or the camera 16). After launching, the camera captures images of the vehicle cabin at regular intervals. In step S14, the in-vehicle system 10 determines whether the brightness in the vehicle cabin is sufficient. The determination of the brightness in the vehicle cabin is performed based on the image of the vehicle cabin captured by the camera launched in step S13. If it is determined that the brightness in the vehicle cabin is insufficient, the in-vehicle system 10 proceeds to step S15. If it is determined that the brightness inside the vehicle compartment is sufficient, the in-vehicle system 10 proceeds to step S16.

[0047] In step S15, the in-vehicle system 10 controls the lighting device 15 to turn it on. In step S16, the in-vehicle system 10 transmits an image of the interior of the vehicle cabin captured by the activated camera 16 to the terminal device 20. That is, if the brightness in the vehicle cabin is insufficient, the in-vehicle system 10 turns on the lighting device 15 and then transmits the image of the interior of the vehicle cabin captured by the camera 16. In step S17, the terminal device 20 receives the image of the interior of the vehicle cabin. In step S18, the terminal device 20 displays the image of the interior of the vehicle cabin on the output device 22.

[0048] Next, the procedure of the vehicle control method in the vehicle control system according to this embodiment will be described with reference to the sequence chart in FIG. 6. FIG. 6 is a sequence chart for explaining the vehicle control method executed by the vehicle control system according to this embodiment. Unlike FIG. 5, the example in FIG. 6 is an example in which the image acquisition instruction by the user includes an instruction for a shooting area. In FIG. 6, when the user presses the start button for an application on the terminal device 20, the terminal device 20 starts the control process from step S20. Note that steps S20 to S23 in FIG. 6 are similar to steps S10 to S13 in FIG. 5, and therefore the description of FIG. 5 will be used as appropriate and omitted here.

[0049] In step S24, the in-vehicle system 10 calculates the display angle of view of the camera 16 activated in step S23. The display angle of view is controlled based on the imaging area instruction included in the image acquisition instruction. In step S25, the in-vehicle system 10 starts a loop process for sequentially controlling the lighting of each lighting device 15. Specifically, the in-vehicle system 10 controls the lighting of lighting device [i] (i = 1 to n) in steps S26 to S29. In step S26, the in-vehicle system 10 calculates an area S where the illumination area of ​​lighting device [i] and the display angle of view overlap. In step S27, the in-vehicle system 10 determines whether the brightness in area S is sufficient. For example, the in-vehicle system 10 acquires illuminance from an image of the interior of the vehicle captured of area S and determines whether the illuminance is equal to or greater than a predetermined threshold. This determination may be performed using luminance instead of illuminance. If it is determined that the brightness in the vehicle interior is insufficient, the in-vehicle system 10 proceeds to step S28. If it is determined that the brightness in the vehicle cabin is sufficient, the in-vehicle system 10 proceeds to step S29. In step S28, the in-vehicle system 10 turns on the lighting device [i]. In step S29, the in-vehicle system 10 turns off the lighting device [i]. In step S30, if the on- and off-control of all lighting devices 15 is complete, the in-vehicle system 10 ends the loop and proceeds to step S31. If the on- and off-control of all lighting devices 15 is not complete, the in-vehicle system 10 returns to step S26, and thereafter repeats the on- and off-control of each lighting device 15 until the loop is completed.

[0050] In step S31, the in-vehicle system 10 transmits an image of the interior of the vehicle captured by the activated camera 16 to the terminal device 20. In step S32, the terminal device 20 receives the image of the interior of the vehicle from the in-vehicle system 10. In step S33, the terminal device 20 displays the image of the interior of the vehicle on the output device 22. In step S34, the terminal device 20 determines whether the display angle of view has changed. If the display angle of view has changed, the in-vehicle system 10 returns to step S24 and repeats the subsequent steps. For example, the terminal device 20 determines that the display angle of view has changed when the user inputs a shooting area instruction to change the shooting area. The terminal device 20 transmits the changed shooting area instruction to the in-vehicle system 10. If the display angle of view has not changed, the terminal device 20 proceeds to step S35. In step S35, the terminal device 20 determines whether to end the display. For example, the terminal device 20 determines that the display should be ended when the user ends the application. If it is determined that the display should be ended, the terminal device 20 proceeds to step S36. If it is determined that the display should not be ended, the terminal device 20 returns to step S32 and repeats the subsequent steps.

[0051] Next, the procedure of the vehicle control method in the vehicle control system of this embodiment will be described based on the sequence chart of FIG. 7 . FIG. 7 is a sequence chart for explaining the vehicle control method executed by the vehicle control system of this embodiment. Unlike FIG. 5 , the example of FIG. 7 is an example in which a left-behind item is detected. When a user presses the application start button on the terminal device 20, the terminal device 20 starts control processing from step S40. Note that steps S40 to S43 and S52 to S54 in FIG. 7 are similar to steps S10 to S13 in FIG. 5 and steps S30 to S32 in FIG. 6 , and therefore their description will be omitted and the description of FIG. 5 will be used as appropriate. Furthermore, after the terminal device 20 displays an image of the vehicle interior in step S54, the terminal device 20 executes the processing described in steps S33 to S35 in FIG. 6 . In FIG. 7 , description of the processing after the terminal device 20 displays the image of the vehicle interior will be omitted, and the specific method will be used as appropriate and the method described in FIG. 6 will be used as appropriate.

[0052] In step S44, the in-vehicle system 10 determines whether or not an item has been left behind in the vehicle cabin. If it is determined that an item has been left behind in the vehicle cabin, the in-vehicle system 10 proceeds to step S45. If it is determined that an item has not been left behind in the vehicle cabin, the in-vehicle system 10 proceeds to step S52. In step S45, the in-vehicle system 10 calculates the angle of view at which the item was detected. In step S46, the in-vehicle system 10 starts a loop process for sequentially controlling the lighting devices to turn on and off. Specifically, in steps S47 to S50, the in-vehicle system 10 controls the lighting device [i] (i = 1 to n) to turn on and off. In step S47, the in-vehicle system 10 calculates an area T where the angle of view at which the item was detected and the lighting area overlap. In step S48, the in-vehicle system 10 determines whether or not area T exists. If it is determined that area T exists, the in-vehicle system 10 proceeds to step S49. If it is determined that the area T does not exist, the in-vehicle system 10 proceeds to step S50. In step S49, the in-vehicle system 10 turns on the lighting device [i]. In step S50, the in-vehicle system 10 turns off the lighting device [i]. In step S51, if the in-vehicle system 10 has completed the on / off control of all lighting devices, it ends the loop and proceeds to step S52.

[0053] As described above, in the vehicle control device and vehicle control method according to this embodiment, the processor determines whether the brightness inside the vehicle cabin is sufficient, and if it determines that the brightness inside the vehicle cabin is insufficient, controls the lighting device to irradiate the vehicle cabin with illumination light, acquires an image of the vehicle cabin taken by a camera after controlling the lighting device, and transmits the acquired image of the vehicle cabin to the outside of the vehicle. This makes it possible to provide a clear image of the vehicle cabin even when the vehicle cabin is dark.

[0054] In the vehicle control device and vehicle control method according to the present embodiment, when a photographing area instruction is received specifying a photographing area to be photographed by a camera in the vehicle cabin, the processor compares the photographing area with the illumination areas that can be irradiated with the illumination light of each of the plurality of lighting devices, identifies a lighting device whose illumination area overlaps the photographing area, and controls the identified lighting device to irradiate the illumination light, thereby making it possible to sufficiently illuminate the photographing area specified by the user.

[0055] In the vehicle control device and vehicle control method according to this embodiment, the processor detects any left-behind item in the vehicle cabin, compares the lighting areas of the lighting devices that can emit light from each of the lighting devices with the location of the left-behind item, identifies the lighting device within whose lighting area the left-behind item is located, and controls the identified lighting device to emit light, thereby enabling a clear image of the vehicle cabin including the left-behind item to be captured.

[0056] In the vehicle control device and vehicle control method according to the present embodiment, when the processor receives an image acquisition instruction to acquire an image of the vehicle interior, the processor activates the camera and determines whether the brightness of the vehicle interior is sufficient based on the image of the vehicle interior captured when the camera is activated. This allows the processor to determine the brightness of the vehicle interior when the user issues an image acquisition instruction.

[0057] In the vehicle control device and vehicle control method according to the present embodiment, the lighting device is a room lamp installed in the vehicle interior, thereby making it possible to sufficiently illuminate the vehicle interior using the room lamp installed in the vehicle interior.

[0058] <<Second Embodiment>> Next, a vehicle control system according to a second embodiment of the present invention will be described. The vehicle control system 1 according to the second embodiment has the same configuration as the vehicle control system 1 shown in FIG. 1 and is similar to the vehicle control system 1 according to the first embodiment, except that it operates as described below. The vehicle control system according to the second embodiment differs from the first embodiment in that it includes an external system (not shown). Unlike the in-vehicle system, the external system is a system located outside the vehicle and not mounted on the vehicle. The external system includes a server and an external communication device. The external system mediates the transmission and reception of information between the in-vehicle system and a terminal device. The server includes a computer having hardware and software. The server includes a ROM storing a program, a CPU that executes the program stored in the ROM, and a RAM that functions as an accessible storage device. The server realizes each of its functions by executing the program stored in the ROM with the CPU.

[0059] The server receives from the terminal device 20, via the exterior communication device, an ID for identifying the terminal device 20, an image acquisition instruction, a shooting area instruction, and the ID of the vehicle for which the user requests an image of the interior of the vehicle. The server references the vehicle ID via the exterior communication device and transmits an image acquisition instruction and a shooting area instruction to the target vehicle. The server receives the vehicle ID and an image of the interior of the vehicle from the in-vehicle system 10 via the exterior communication device. The server transmits the image of the interior of the vehicle to the terminal device 20 that has requested the image of the interior of the vehicle. The exterior communication device is a communication interface compatible with various communication standards such as wireless LAN standards. The exterior communication device is a device for exchanging information between the vehicle control device 11 and the terminal device 20 via a network.

[0060] It should be noted that the above-described embodiments have been described to facilitate understanding of the present invention, and are not intended to limit the present invention. Therefore, the elements disclosed in the above embodiments are intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

[0061] REFERENCE SIGNS LIST 1 vehicle control system 10 in-vehicle system 11 vehicle control device 100 processor 12 in-vehicle communication device 13 storage 14 illuminance acquisition device 15 lighting device 16 camera 20 terminal device

Claims

1. A vehicle control device having a processor, wherein the processor determines whether the brightness inside the vehicle interior is sufficient, and if it determines that the brightness inside the vehicle interior is not sufficient, controls a lighting device to irradiate the interior with illumination light, acquires an image of the interior of the vehicle interior taken by a camera after controlling the lighting device, and transmits the acquired image of the interior of the vehicle interior to an outside of the vehicle.

2. A vehicle control device as described in claim 1, wherein, when the processor receives a shooting area instruction indicating the shooting area to be photographed by the camera within the vehicle cabin, the processor compares the lighting areas that can be irradiated with the illumination light of each of the plurality of lighting devices with the shooting area, identifies the lighting device of the plurality of lighting devices whose lighting area overlaps with the shooting area, and controls the identified lighting device to emit the illumination light.

3. A vehicle control device as claimed in claim 1 or 2, wherein the processor detects an item left behind in the vehicle interior, compares the lighting area in which the lighting light of each of the plurality of lighting devices can be irradiated with the position of the item, identifies the lighting device among the plurality of lighting devices whose lighting area contains the item, and controls the identified lighting device to emit the lighting light.

4. A vehicle control device according to any one of claims 1 to 3, wherein the processor activates the camera when it receives an image acquisition instruction to acquire an image of the interior of the vehicle, and determines whether the brightness of the interior of the vehicle is sufficient based on the image of the interior of the vehicle captured when the camera is activated.

5. A vehicle control device according to any one of claims 1 to 4, wherein the lighting device is a room lamp installed inside the vehicle compartment.

6. A vehicle control method executed by a processor, wherein the processor determines whether the brightness inside the vehicle interior is sufficient, and if it determines that the brightness inside the vehicle interior is not sufficient, controls a lighting device to irradiate illumination light into the vehicle interior, acquires an image of the vehicle interior taken by a camera after controlling the lighting device, and transmits the acquired image of the vehicle interior to the outside of the vehicle.

Images