In-vehicle monitoring system

Abstract

To provide an "in-vehicle monitoring system" that reduces the range of an area that cannot be monitored from an image captured by an in-vehicle camera.SOLUTION: An automobile structure with high reflectance, such as a door mirror, a mirror-finished decorative part, and a metal part, the eyes of a passenger, and an accessory and luggage with high reflectance of a passenger are set as reflecting objects. An area that cannot be captured by an in-vehicle camera 2 is calculated as a non-capturable area, and areas where the images reflected on the reflecting objects in the image captured by the in-vehicle camera 2 are reflected are calculated as reflected image ranges. An area in the reflected image ranges where a reflected image is reflected is calculated as an area to be reflected. An area where the non-capturable area and the area to be reflected overlap each other is set as a complementary area, and an image portion of the complementary area reflected on the reflecting objects in the image captured by the in-vehicle camera 2 is used as an image of the complementary area to monitor the interior of a vehicle.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a technique for monitoring the interior of a vehicle using video captured by a camera. [Background technology]

[0002] Known technologies for monitoring the interior of a vehicle using video captured by a camera include a technology that records video captured by a camera placed in the position of the rearview mirror (for example, Patent Document 1), and a technology that detects whether or not an item has been left behind in the vehicle by checking whether or not there is any change in the state of the interior of the vehicle captured by the camera between when the passenger gets in and out (for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2001-119687 A [Patent Document 2] JP 2006-338535 A Summary of the Invention [Problem to be solved by the invention]

[0004] When monitoring the interior of a vehicle using footage captured by a camera, it is not possible to monitor blind spots that are hidden from the camera by seats, occupants, or luggage, or areas inside the vehicle that are outside the camera's field of view, such as areas that cannot be captured by the camera.

[0005] Placing multiple cameras in different positions can reduce the overall unviewable area, but this leads to increased costs. Also, when the positions where cameras can be placed are limited, it may not be possible to sufficiently reduce the unviewable area. Even if multiple cameras are placed so that there is no unviewable area when using a standard seat arrangement, blind spots may still occur as unviewable areas when the seat arrangement is changed, etc.

[0006] Therefore, an object of the present invention is to reduce the range of areas that cannot be monitored from images captured by cameras without using multiple cameras. [Means for solving the problem]

[0007] In order to achieve the above object, the present invention provides an interior monitoring system for monitoring the interior of a vehicle, which includes a camera for capturing images of the interior of the vehicle, an uncaptureable area setting means for setting at least a portion of the interior of the vehicle that cannot be captured by the camera as an uncaptureable area, a reflected image range setting means for setting the area within the captured image, which is the image captured by the camera, in which an image reflected by components of the vehicle appears as a reflected image range, a reflected area setting means for setting the area within the vehicle interior in which a reflected image appears in the reflected image range as a complementary area, a complementary area setting means for setting an area within the reflected area that includes at least a portion that overlaps with the uncaptureable area as a complementary area, and a complementary image setting means for setting the image of the complementary area reflected by the components within the reflected image range of the captured image as a complementary image, which is an image showing the complementary area.

[0008] Here, the automobile component may include at least one of an automobile rearview mirror, an automobile interior part, an automobile metal part, and an automobile windshield.

[0009] In order to achieve the above object, the present invention provides an in-vehicle monitoring system for monitoring the condition inside a vehicle, which includes a camera for capturing images of the interior of the vehicle, an uncaptureable area setting means for setting at least a portion of the interior of the vehicle that cannot be captured by the camera as an uncaptureable area, a reflected image range setting means for setting the area in which an image reflected by the pupils of the vehicle occupants appears in the captured image, which is the image captured by the camera, as a reflected area, a reflected area setting means for setting an area within the reflected area that includes at least a portion that overlaps with the uncaptureable area as a complementary area, and a complementary image setting means for setting the image of the complementary area captured by the pupils within the reflected image range of the captured image as a complementary image, which is an image showing the complementary area.

[0010] In order to achieve the above object, the present invention provides an interior monitoring system for monitoring the interior of a vehicle, the system being provided with a camera for capturing images of the interior of the vehicle, a non-capturing area setting means for setting at least a part of the interior of the vehicle that cannot be captured by the camera as a non-capturing area, a reflected image range setting means for setting a range in which an image reflected by an object brought into the vehicle by an automobile occupant appears in a captured image captured by the camera as a reflected image range, a reflected area setting means for setting an interior of the vehicle where an image reflected in the reflected image range appears as a reflected area, a complementary area setting means for setting an area in the reflected area that includes at least a portion that overlaps with the non-capturing area as a complementary area, and a complementary image setting means for setting an image of the complementary area reflected by the object within the reflected image range of the captured image as a complementary image that represents the complementary area. Here, the object includes at least one of an accessory worn by the occupant, a portable object of the occupant, and a baggage of the occupant.

[0011] In order to achieve the above-mentioned object, the present invention provides an in-vehicle monitoring system for monitoring the situation inside a vehicle, which includes a camera for capturing images of the interior of the vehicle, an unphotographable area setting means for setting at least a portion of the area inside the vehicle that cannot be captured by the camera as an unphotographable area, a reflected image range setting means for analyzing the captured image, which is the image captured by the camera, and calculating a reflected image range in which an image reflected by a reflective object appears, a reflected area setting means for calculating the area inside the vehicle in which an image reflected in the reflected image range appears as a reflected area, a complementary area setting means for setting an area within the reflected area that includes at least a portion that overlaps with the unphotographable area as a complementary area, and a complementary image setting means for setting an image of the complementary area reflected by personal belongings within the reflected image range of the captured image as a complementary image, which is an image showing the complementary area.

[0012] Here, in such an in-vehicle monitoring system, the reflected image range setting means may analyze the captured image, which is the image captured by the camera, and calculate the reflected image range, which is the range in which an image that changes in response to the movement of the occupants in the captured image appears and which is different from the range in which an image that is not due to the reflection of the movement appears, as the reflected image range, which is the range in which an image due to reflection of a reflecting object appears.

[0013] In addition, in such an in-vehicle monitoring system, the reflected area setting means may calculate the reflected area based on at least one of the position of the reflecting object calculated based on the reflected image range and a position in the captured image where an image identical to the image in the reflected image range should appear if the reflected area is not blocked from the camera, which is determined from a part of the captured image outside the reflected image range where an image overlaps or is continuous with an image in the reflected image range in the captured image.

[0014] In addition, the above-mentioned in-vehicle monitoring system may be provided with a blind spot area calculation means that detects the state of an object inside the vehicle and calculates, based on the state of the detected object, an area that is blocked from the camera by the object inside the vehicle as a blind spot area, and the unphotographable area setting means may set the unphotographable area so as to include the blind spot area.

[0015] Furthermore, the above-mentioned in-vehicle monitoring system may be provided with a monitoring means for performing a predetermined monitoring process for monitoring the interior of the vehicle using the captured image, and the monitoring means may be configured to monitor the complementary area using the complementary image during the monitoring process.

[0016] With the above-described in-vehicle monitoring system, in addition to monitoring areas where images can be directly captured by the camera, it is possible to monitor areas that cannot be captured, such as areas outside the camera's angle of view, which are areas outside the capture range, and blind spots, which are areas inside the vehicle that are hidden from the camera by seats, occupants, or luggage, in a manner that does not incur a significant increase in cost by at least partially utilizing reflections from existing reflective objects in the vehicle, such as the vehicle's rearview mirror, interior parts, metal parts, and other components, the eyes of occupants, and items brought into the vehicle by occupants. Effect of the Invention

[0017] As described above, according to the present invention, it is possible to reduce the range of areas that cannot be monitored from images captured by the cameras without using multiple cameras. [Brief description of the drawings]

[0018] [Figure 1] 1 is a block diagram showing a configuration of an in-vehicle monitoring system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a diagram showing the arrangement of an in-vehicle camera according to the embodiment of the present invention. [Diagram 3] 10 is a flowchart showing a complementary setting calculation process according to an embodiment of the present invention. [Figure 4] 1A to 1C are diagrams illustrating examples of reflectors used in an embodiment of the present invention. [Diagram 5] 5 is a flowchart showing a surveillance video generating process according to the embodiment of the present invention. [Figure 6] 1A and 1B are diagrams illustrating examples of areas that can be monitored from reflected images in an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present invention will be described. FIG. 1 shows the configuration of an in-vehicle monitoring system. The in-vehicle monitoring system is a system mounted on an automobile, and comprises a status sensor 1, an in-vehicle camera 2, a monitoring image generating unit 3, a complementary setting calculating unit 4, and a monitoring image utilizing system 5, as shown in the figure.

[0020] The state sensor 1 is a sensor that detects various states of the automobile, such as the position of the automobile seat and the angle of the seat backrest. The in-car camera 2 is an in-car camera 2 that captures the state inside the car, and the in-car camera 2 captures the rearward direction, for example, as shown in Figures 2a and 2b. Here, the in-car camera 2 may be an infrared in-car camera or a stereo camera capable of three-dimensional measurement.

[0021] 1, the complementary setting calculation unit 4 calculates complementary settings for the surveillance video generation unit 3, and the surveillance video generation unit 3 generates surveillance video showing the inside of the vehicle from the video captured by the in-vehicle camera 2 in accordance with the complementary settings. The complementary settings and the operation of generating surveillance video in accordance with the complementary settings will be described in detail later.

[0022] The surveillance video utilization system 5 is a system that performs surveillance processing using the surveillance video output by the surveillance video generation unit 3. The surveillance processing includes, for example, recording and playing back surveillance video, detecting lost items, detecting the status of occupants in the vehicle, and detecting the occurrence of a specified event in the vehicle.

[0023] Next, the operation of the complementary setting calculation unit 4 will be described. The complementary setting calculation unit 4 constantly stores images captured by the in-vehicle camera 2 for a certain period of time in the recent past. Furthermore, the complementary setting calculation unit 4 periodically repeats the complementary setting calculation process. FIG. 3 shows the procedure for this complementary setting calculation process. As shown in the figure, in the complementary setting calculation process, the complementary setting calculation unit 4 first calculates areas outside the capture range, which are areas outside the angle of view of the in-vehicle camera 2, and blind spot areas, which are areas inside the vehicle that are hidden from the in-vehicle camera 2 by seats, passengers, or luggage, as non-capturing areas, which are areas that cannot be captured by the in-vehicle camera 2 (step 302).

[0024] Here, the area outside the shooting range of the in-vehicle camera 2 is set in advance according to the shooting range of the in-vehicle camera 2. Also, the blind spot area is calculated from, for example, the range that objects determined from the position of the car seat and the angle of the seat back detected by the status sensor 1, and the positions and shapes of the seats, passengers, and luggage identified by image recognition processing of the image captured by the in-vehicle camera 2 occupy in relation to the angle of view of the in-vehicle camera 2, and the structure of the vehicle interior.

[0025] Next, the image captured by the in-vehicle camera 2 is analyzed, and a reflected image range, which is the range of the reflecting object in the captured image, is calculated (step 304). A reflective object is an object within the shooting range of the in-vehicle camera 2 that has a high reflectivity such that a reflected image can be identified from the image captured by the in-vehicle camera 2, and includes decorative parts and metal parts such as door mirror 401 shown in FIG. 4a, door trim 402 with a mirror finish such as piano black color shown in FIG. 4b, eyes 403 of an occupant shown in FIG. 4c, accessories 404 with a high reflectivity such as jewels and precious metals worn by an occupant, and luggage 405 of an occupant such as a hand mirror. Also, at night, a side windshield 406 of an automobile at night shown in FIG. 4d is also a reflective object. However, if the in-vehicle camera 2 is an infrared in-vehicle camera and the side windshield 406 is infrared-blocking glass, the windshield will be a reflective object even during the day.

[0026] The reflected image range is calculated, for example, as follows. In other words, the occurrence of movements by occupants is detected from the stored images captured by the in-vehicle camera 2 within a certain period of time in the past, and the range of the area in the image where an image similar to the detected movement occurs in synchronization with the detected movement is calculated as the reflected image range.

[0027] However, for fixed reflecting objects such as door mirrors, decorative parts of an automobile, metal parts, and windshields, the reflected image range may be obtained and registered in advance, and the registered reflected image range may be used in step 304.

[0028] For reflecting objects that are the eyes of the occupants, the range of the eyes of the occupants in the video captured by the in-vehicle camera 2 may be image-recognized, and the range of the recognized image of the eyes may be set as the reflected image range. For other reflecting objects, the reflecting objects in the video captured by the in-vehicle camera 2 may be image-recognized, and the range of the recognized image of the reflecting object may be set as the reflected image range.

[0029] Next, it is checked whether the reflected image range has been detected (step 306), and if not, the complement setting is set to no complement in the surveillance image generating unit 3 (step 320), and the complement setting calculation process is terminated.

[0030] On the other hand, if the reflected image range can be detected (step 306), the attributes of each reflector whose reflected image range has been detected are calculated, and the calculated attributes are registered as reflector attributes (step 308). Here, the attributes of reflectors include "fixed", which is an attribute of a reflector that is fixed to the automobile, "movable", which is an attribute of a reflector that is movable to the automobile, "constant", which is an attribute of a reflector that always functions as a reflector, and "indefinite", which is an attribute of a reflector that sometimes functions as a reflector and sometimes does not function as a reflector.

[0031] Here, the attributes of the reflecting object are calculated by identifying the type of reflecting object from the image of the reflected image range of the video captured by the in-vehicle camera 2 and determining the attributes that are previously associated with the type. However, for fixed reflecting objects such as door mirrors, decorative parts of an automobile, metal parts, and windshields, the corresponding attributes may be registered in advance, and the registered attributes may be used in step 308.

[0032] Next, for each reflected image area, the reflected area is calculated (step 310). The reflected area is an area inside the vehicle whose image is captured in the reflected image range, that is, an area inside the vehicle whose image is captured by the in-vehicle camera 2 due to reflection of a reflecting object corresponding to the reflected image range.

[0033] The reflected area is calculated from the positional relationship between the position of the reflecting object corresponding to the reflected image range and the position of the in-vehicle camera 2, and the overlap and continuity of the shapes of the images within the reflected image range of the image captured by the in-vehicle camera 2 and other parts of the image captured by the in-vehicle camera 2.

[0034] Here, for a reflector with the attribute "fixed", the position of the reflector may be calculated from the direction of the reflector relative to the in-car camera 2, which is determined from the position in the image of the reflected image range, and the structure of the vehicle interior, or the position of the reflector may be measured and registered in advance. Also, for a reflector with the attribute "movable", the position of the reflector is calculated as an approximate position from the direction of the reflector relative to the in-car camera 2, which is determined from the position in the image of the reflected image range, the state of the seat, such as the position of the seat and the angle of the seat back, detected by the state sensor 1, and the structure of the vehicle interior.

[0035] The position of the reflecting object may also be calculated by directly measuring the position using a stereo camera capable of three-dimensional measurement as the in-vehicle camera 2. In addition, the reflected area can be calculated from the overlap or continuity of the shapes of the images within the reflected image range of the image captured by the in-vehicle camera 2 and other parts of the image captured by the in-vehicle camera 2, for example by determining other parts of the image captured by the in-vehicle camera 2 whose shapes overlap or are continuous with the images within the reflected image range, calculating a range in the image captured by the in-vehicle camera 2 where an image the same as the image within the reflected image range should appear if there were no obstructions from the shape of the overlap or continuity between the determined part and the image within the reflected image range, determining the direction of the reflected area relative to the in-vehicle camera 2 from the position of the calculated range in the image, and calculating the reflected area from the determined direction and the structure of the automobile interior.

[0036] Next, the area where the reflection area calculated in step 310 and the non-captureable area calculated in step 302 overlap is set as a complement area (step 312). Then, it is checked whether there is a complementary area (step 314), and if there is no overlap between the reflected area and the non-capturing area and there is no complementary area, the complement setting is set to no complement in the surveillance image generating unit 3 (step 320), and the complement setting calculation process is terminated.

[0037] On the other hand, if there is a complementary area that is an overlapping portion between the reflected area and the non-photographable area (step 314), the area within the reflected image range in which the image due to reflection of the reflecting object in the complementary area appears is set as the complementary image range (step 316).

[0038] Then, each pair of the complement image range and the complement area obtained for each complement area is set as a complement setting in the surveillance video generating unit 3 (step 318), and the complement setting calculation process ends. The complementary setting calculation process performed by the complementary setting calculation unit 4 has been described above. Here, in the above-described complementary setting calculation process, in step 312, the entire reflected area may be set as the complementary area, and in step 316, the entire reflected image area may be set as the complementary image area.

[0039] In addition, in the above-described complementary setting calculation process, when a reflector with attributes of “fixed” and “always” is registered, the processes of steps 304, 308-312, and 316 may be omitted for that reflector, and in step 318, the complementary area and complementary image range previously calculated for that reflector may be used.

[0040] Next, the surveillance video generation process performed by the surveillance video generation unit 3 will be described. FIG. 5 shows the procedure for this surveillance video generation process. As shown in the figure, in the surveillance video generation process, the surveillance video generation unit 3 determines whether the complement setting is no complement (step 502), and if no complement is set, outputs the video captured by the in-vehicle camera 2 as surveillance video to the surveillance video utilization system 5 (step 510). Then, the process returns to step 502.

[0041] When only the image captured by the in-vehicle camera 2 is output as the surveillance image from the surveillance image generating unit 3, the surveillance image utilization system 5 performs predetermined surveillance processing on the image captured by the in-vehicle camera 2, such as recording and reproducing the image, detecting items left behind, detecting the status of passengers in the vehicle, and detecting the occurrence of a predetermined event in the vehicle. On the other hand, if the complement setting is not no complement (step 502), an image of the complement image range in the image captured by the in-vehicle camera 2 is extracted for each complement setting (step 504), and for each complement setting, a viewpoint transformation process such as an affine transformation according to the positional relationship between the complement area and the in-vehicle camera 2 is performed to transform the image of the complement image range extracted for that complement setting into a viewpoint transformation image, which is an image obtained when the complement area is photographed from the position of the in-vehicle camera 2 (step 506).

[0042] Then, the viewpoint converted video generated for each complementation setting and the video captured by the in-vehicle camera 2 are output as a surveillance video to the surveillance video utilization system 5 (step 508). When both the viewpoint converted video and the video captured by the in-vehicle camera 2 are output as surveillance video from the surveillance video generating unit 3, the surveillance video utilization system 5 performs a predetermined surveillance process on both the viewpoint converted video and the video captured by the in-vehicle camera 2. Therefore, in this case, the surveillance video utilization system 5 also performs surveillance processing on the complementary area, which is at least a part of the area that cannot be directly captured by the in-vehicle camera 2.

[0043] The surveillance video generation process performed by the surveillance video generator 3 has been described above. In addition, the processing performed by the surveillance image generation process when the complement setting is not no complement may be other processing, as long as it outputs the image captured by the in-vehicle camera 2 and the image of the complemented image range in the image captured by the in-vehicle camera 2 in a form that makes it possible to identify which area within the vehicle the image of the complemented image range represents.

[0044] For example, the surveillance video generation process may be performed when the complement setting is not no complement, by outputting the video captured by the in-vehicle camera 2, an image of the complemented image range in the video captured by the in-vehicle camera 2, and information on the complemented area to the surveillance video utilization system 5.

[0045] Alternatively, the surveillance video generation process may be performed when the complement setting is not no complement by converting both the image captured by the in-vehicle camera 2 and the images of each complement image range in the image captured by the in-vehicle camera 2 into images that would be obtained when captured from a specified viewpoint position set above the vehicle cabin, combining each of the converted images into a single image that shows the interior of the vehicle as observed from a viewpoint above the vehicle cabin, and outputting the combined image to the surveillance video utilization system 5 as surveillance video.

[0046] As described above, with this in-vehicle monitoring system, in addition to monitoring areas where images can be directly captured by the in-vehicle camera 2, it is possible to at least partially monitor areas outside the capture range, which is areas outside the angle of view of the in-vehicle camera 2, and blind spots, which are areas inside the vehicle that are hidden from the in-vehicle camera 2 by seats, occupants, or luggage, in a manner that does not incur a significant increase in cost by utilizing the reflection of reflectors already in the automobile.

[0047] That is, for example, as shown in FIG. 6a, it is possible to use images reflected by the pupils of the front seat occupants to monitor the space 601 in front of and below the front seats, which is an area outside the range of the in-vehicle camera 2, or as shown in FIG. 6b, it is possible to use images reflected by the side windows and door trim to monitor the area around the rear seat cushion and the space 602 in front of and below the rear seats, which are blind spots blocked by the front seats for the in-vehicle camera 2.

[0048] In the above embodiment, only one in-vehicle camera 2 is provided on the dashboard, but the number and arrangement of the in-vehicle cameras 2 may be different. [Explanation of symbols]

[0049] 1...status sensor, 2...in-vehicle camera, 3...surveillance image generation unit, 4...complementary setting calculation unit, 5...surveillance image utilization system.

Claims

1. An in-vehicle monitoring system for monitoring the status inside an automobile, comprising: A camera that records the inside of the car, a capture-prohibited area setting means for setting at least a part of an area inside the vehicle that cannot be captured by the camera as a capture-prohibited area; a reflected image range setting means for setting a range in which an image reflected by the pupils of an automobile occupant appears in a captured image captured by the camera as a reflected image range; a reflection area setting means for setting an area inside the vehicle in which a reflected image appears in the reflection image range as a reflection area; a complementary area setting means for setting an area including at least a portion of the reflection area that overlaps with the non-photographable area as a complementary area; An in-vehicle monitoring system characterized by having a complementary image setting means for setting the image of the complementary area caused by reflection of the pupil within the reflection image range of the captured image as a complementary image which is an image representing the appearance of the complementary area.

2. An in-vehicle monitoring system for monitoring the status inside an automobile, comprising: A camera that records the inside of the car, a capture-prohibited area setting means for setting at least a part of an area inside the vehicle that cannot be captured by the camera as a capture-prohibited area; a reflected image range setting means for setting a range in which an image reflected by an object brought into the vehicle by an occupant of the vehicle appears in the captured image captured by the camera as a reflected image range; a reflection area setting means for setting an area inside the vehicle in which a reflected image appears in the reflection image range as a reflection area; a complementary area setting means for setting an area including at least a portion of the reflection area that overlaps with the non-photographable area as a complementary area; a complementary image setting means for setting an image of the complementary area reflected by the object within the reflected image range of the photographed image to a complementary image which is an image representing the complementary area; The object includes at least one of an accessory worn by the occupant, a personal belonging of the occupant, and luggage of the occupant.

3. An in-vehicle monitoring system for monitoring a condition inside an automobile, comprising: A camera that records the inside of the car, a capture-prohibited area setting means for setting at least a part of an area inside the vehicle that cannot be captured by the camera as a capture-prohibited area; A reflected image range setting means for analyzing a captured image captured by the camera and calculating a reflected image range in which an image reflected by a reflecting object appears; a reflection area setting means for calculating an area inside the vehicle in which a reflected image appears in the reflection image range as a reflection area; a complementary area setting means for setting an area including at least a portion of the reflection area that overlaps with the non-photographable area as a complementary area; An in-vehicle monitoring system characterized by having a complementary image setting means for setting the image of the complementary area reflected by the belongings of the vehicle occupants within the reflected image range of the captured image as a complementary image that represents the appearance of the complementary area.

4. The in-vehicle monitoring system according to claim 3, The reflected image range setting means of this in-vehicle monitoring system calculates the reflected image range as a range in which an image that changes in response to the movements of the vehicle occupants in the captured image appears, the range being different from the range in which an image that is not a reflection of the movements appears.

5. An in-vehicle monitoring system according to claim 3 or 4, The reflection area setting means calculates the position of the reflecting object based on the reflection image range, a position in the captured image where an image overlapping or continuous with the image in the reflected image range in the captured image appears, the position being determined from a portion outside the reflected image range in the captured image, and where an image identical to the image in the reflected image range should appear if the reflected area is not shielded from the camera; The in-vehicle monitoring system is characterized in that the reflection area is calculated based on at least one of the above.

6. An in-vehicle monitoring system according to claim 1, 2, 3, 4 or 5, a blind spot area calculation means for detecting a state of an object inside the vehicle and calculating an area blocked by the object inside the vehicle from the camera as a blind spot area based on the state of the detected object; The in-vehicle monitoring system is characterized in that the imaging-prohibited area setting means sets the imaging-prohibited area so as to include the blind spot area.

7. An in-vehicle monitoring system according to claim 1, 2, 3, 4, 5 or 6, a monitoring means for performing a predetermined monitoring process for monitoring the interior of the vehicle using the captured image, The monitoring means monitors the complemented area by using the complemented image in the monitoring process.

Images