Camera system, installation method, and computer program

Abstract

To provide a camera system that can simultaneously capture an area of interest under a specified condition using a single wide-angle camera.SOLUTION: A camera system has a wide-angle camera arranged in the passenger compartment of a vehicle, the wide-angle camera is arranged between the driver's seat and the rear seat with its optical axis facing downward, and arranged such that the area of interest when the vehicle is moving and the area of interest when the vehicle is stopped are within the range of φ / 2 to φ, with the angle φ being the viewing angle of the wide-angle camera.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an in-vehicle camera system, an installation method, a computer program, and the like. [Background technology]

[0002] Conventionally, there are known in-vehicle cameras that observe the inside of the vehicle while the vehicle is moving to observe the driver's condition and the condition of the passengers. There are also known cameras that simultaneously observe the inside of the vehicle and the driving condition outside the vehicle while the vehicle is moving, and monitor people who are outside the vehicle while the vehicle is stopped and who may harm the vehicle or who may break into the vehicle.

[0003] There is also a known technology for using a vehicle interior camera for multiple purposes. For example, Patent Document 1 discloses a technology for capturing images for recognition inside the vehicle interior with high sensitivity while driving, and capturing images for recognition outside the vehicle interior with low sensitivity.

[0004] Patent Document 2 discloses a technology that divides a captured image into an area inside the vehicle cabin and an area outside the vehicle cabin, obtains the brightness of the area to be observed, controls the imaging conditions for each area, and records the image in a drive recorder. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2022-102525 [Patent Document 2] JP 2012-228931 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, the conventional technology disclosed in the above-mentioned Patent Document 1 does not specify the camera's angle of view, and does not mention surroundings monitoring while the vehicle is stopped. Therefore, it is unclear how many camera systems are required to monitor the surroundings while the vehicle is stopped.

[0007] In addition, the conventional technology disclosed in Patent Document 2 does not design a camera system that takes into consideration monitoring the entire periphery outside the vehicle cabin, and the area of ​​interest inside the vehicle cabin and the area of ​​interest outside the vehicle cabin are separated within the camera's viewing angle. Therefore, each area requires high resolution to achieve the purpose. For this reason, an expensive image sensor with high resolution is required.

[0008] Therefore, one object of the present invention is to provide a camera system capable of simultaneously capturing an image of an area of ​​interest under a given condition with a single wide-angle camera. [Means for solving the problem]

[0009] In order to achieve the above object, a camera system according to one aspect of the present invention comprises: A wide-angle camera is disposed in the interior of the vehicle, The wide-angle camera is disposed between the driver's seat and the rear seat with its optical axis facing downward, The area of ​​interest when the vehicle is moving and the area of ​​interest when the vehicle is stopped are arranged so as to be within a range of φ / 2 to φ, with respect to the angle φ as the viewing angle of the wide-angle camera. It is characterized by: Effect of the Invention

[0010] According to the present invention, it is possible to provide a camera system capable of simultaneously capturing an image of an area of ​​interest under a predetermined condition using a single wide-angle camera. [Brief description of the drawings]

[0011] [Figure 1] 1A and 1B are diagrams illustrating an example of the arrangement of a wide-angle camera according to the first embodiment of the present invention. [Diagram 2] 1A and 1B are diagrams for explaining an image captured by a wide-angle camera inside a vehicle according to a first embodiment of the present invention. [Diagram 3]4 is a diagram showing image height characteristics of a fisheye lens in the optical system according to the first embodiment, where the angle of view from the optical axis is from 0 degrees to 90 degrees. FIG. [Figure 4] 1A and 1B are diagrams showing an example of the relationship between a camera and a target of interest (region of interest) according to the first embodiment. [Diagram 5] 4A to 4C are diagrams illustrating an example of camera arrangement and an example of a determination result in the first embodiment. [Figure 6] 11A and 11B are diagrams illustrating an example of a camera arrangement and an example of a determination result in Comparative Example 1 which is different from the first embodiment. [Figure 7] FIG. 11 is a diagram showing an example of a comparative example 2 having image height characteristics different from those of the first embodiment. [Figure 8] 13 is a flowchart showing an example of an operating condition change process during shooting in the third embodiment. [Figure 9] 13(A) and 13(B) are diagrams for explaining an example of a region of interest in a camera system according to a fifth embodiment. [Figure 10] FIG. 13 is a diagram showing an example of a relationship between a camera and an object of interest according to the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions are omitted or simplified.

[0013] <Embodiment 1> 1(A) and (B) are diagrams for explaining an example of the arrangement of a wide-angle camera according to the first embodiment of the present invention. Fig. 1(A) is a vertical cross-sectional view of an automobile, and (B) is a diagram showing an example of an image captured by the camera.

[0014] In the figure, 1 indicates an automobile as a moving object, 3 indicates a wide-angle camera installed inside the automobile's cabin, 5 indicates the peripheral visual field, 6 indicates the peripheral visual field angle boundary, 7 indicates the driver, 9 indicates the driver's head, 11 indicates the steering wheel, 13 indicates the driver's hands, 15 indicates the driver's feet, 17 indicates the passengers in the rear seats, 19 indicates the heads of the passengers in the rear seats, 21 indicates the front window, 23 indicates the side windows of the driver's seat and the passenger seat, 25 indicates the rear seat side window, and 27 indicates the rear window.

[0015] 1 shows an example of a sedan with two rows of seats, but it may be a station wagon or the like. The rear seats may also have two or three rows. A microbus or the like having even more rear seats may also be used, but in order to suitably apply this embodiment, it is preferable for the seat to have two or three rows. That is, with up to three rows of seats, the heads 19 of the passengers in the rear seats can be observed well.

[0016] The wide-angle camera 3 is a camera having a viewing angle of 90 degrees or more and a wide-angle lens such as a fisheye lens. That is, the wide-angle camera 3 of this embodiment includes a fisheye camera. The wide-angle camera 3 has a lens, an image sensor, a digital signal processing unit, a communication unit, etc.

[0017] The lens focuses an image outside the camera onto the light receiving surface of the image sensor, where the light information is converted into an analog voltage signal, which is then converted into a digital signal to become digital image data. The digital image data is processed in the digital signal processing unit and is either transmitted to the outside as image data or recorded.

[0018] The image data is displayed on a monitor as a display means and is used by the driver to make decisions. Image recognition processing is then performed, and the recognition results are used for autonomous driving and driving assistance. The image data is also used directly in real time. Furthermore, by saving it to a storage device, it can be used to check for hit-and-run accidents from surrounding vehicles while driving, check for information on road obstructions, and check the damage situation such as car break-ins while parked.

[0019] Reference numeral 5 denotes the peripheral visual field. If the optical axis is 0 degrees and the angle from there to the edge of the field of view (FOV) where the edge of the image is captured is φ, the peripheral visual field 5 is φ / 2 to φ. In the case of a fisheye lens that is 90 degrees on one side from the optical axis, when the optical axis is 0 degrees, the peripheral visual field refers to the area from 45 degrees to 90 degrees.

[0020] Numeral 6 denotes the peripheral viewing angle boundary, and the boundary at φ / 2 is defined as the peripheral viewing angle boundary. When the image captured by the image sensor is observed without distortion correction, this boundary is shown as a circle like 6 in Figure 1(B). When viewed as a cross-sectional image, it is shown as a dashed line like Figure 1(A).

[0021] The driver 7 is seated on the right side in FIG. 1(B), but if the vehicle has a steering wheel on the left side, the driver may be seated on the left side.

[0022] Reference numeral 9 denotes the driver's head. The position of the driver's head changes depending on the posture of the driver 7, but in most cases the head is in front of the headrest, so in this embodiment the position of the driver's head in front of the headrest is taken as the position of the driver's head 9. Also, if there is no headrest, the position of the eyes may be taken as the position of the driver's head 9, since the position of the eyes is specified for each vehicle during the type approval test.

[0023] Reference numeral 11 denotes a steering wheel, which functions to steer the vehicle forward and backward. In FIG. 1(B), the steering wheel 11 is located on the right side of the vehicle, but it may be located on the left side. It may be circular or non-circular. In this embodiment, even a part of the steering wheel 11 can be observed with high resolution, and therefore information on the steering state can be obtained, so the steering wheel 11 may be a part of the steering wheel.

[0024] Reference numeral 13 denotes the driver's hand, which transmits the person's decision to the steering wheel when steering. As with the steering wheel, in this embodiment, even a part of the driver's hand 13 can be observed with high resolution, so information on the steering state can be obtained. Therefore, the driver's hand 13 may be a part of the hand. Note that the driver may be wearing gloves, or may be completely hidden by clothing, in which case the assumed position of the hand is taken as the position of the driver's hand 13.

[0025] Reference numeral 15 denotes the driver's feet. They are mainly intended to step on the accelerator and brake when driving, and are the subject of the region of interest. In actual observation, the feet 15 are often shoes. As with the driver's hands 13, even a part of the driver's feet 15 is treated as the driver's feet 15 in this case. Also, if the feet are completely obscured by clothing, the assumed position of the feet is taken as the position of the driver's feet 15.

[0026] In particular, although the driver can move the feet laterally with some freedom, the main concern is the operation of the accelerator and brake, so the lateral position of the feet is determined based on the center line of the body in the lateral direction.

[0027] Reference numeral 17 denotes rear seat passengers. These passengers sit behind the driver's seat, and Fig. 1(B) shows a typical two-row sedan with two rear seat passengers 17 sitting in the rear seats. However, in the case of a vehicle with multiple rows of seats, such as three rows, passengers sitting in the multiple rows of seats behind the driver's seat are considered rear seat passengers. Also, people sitting in child seats or junior seats may be considered rear seat passengers.

[0028] Reference numeral 19 denotes the head of the rear seat passenger. This is the location of the head of the rear seat passenger 17, and like the driver 7, the head position changes depending on the posture, but since the head is positioned in front of the headrest most of the time, the head positioned in front of the headrest is treated as the rear seat passenger's head 19.

[0029] Next, an example of the positional relationship between the camera and the region of interest according to this embodiment will be described with reference to Fig. 2. Figs. 2(A) and (B) are diagrams for explaining an image captured by the wide-angle camera inside the vehicle according to the first embodiment of the present invention. Fig. 2(A) is a diagram of an image capturing the region of interest while driving, and Fig. 2(B) is a diagram of an image capturing the region of interest while parking. In Figs. 2(A) and (B), the region indicated by diagonal lines is the region of interest.

[0030] The shaded area in Fig. 2(A) is the area of ​​interest while driving, and includes the driver's physical condition and the conditions of passengers in the back seats. In other words, the area of ​​interest while driving is the area including the heads of people inside the vehicle. In particular, since the driver's physical condition is directly related to driving safety, an area with a relatively high resolution is allocated so that facial expressions can be recognized to detect drowsiness, etc.

[0031] In this embodiment, the presence or absence of operational errors during driving can be recorded by arranging the steering wheel 11, hands 13, and feet 15 so that they are visible as areas of interest. The area ahead in the direction of travel of the car is also considered to be an area of ​​interest while the car is traveling.

[0032] It is desirable to use the image of the region of interest acquired during driving for automatic driving, drowsiness alerts, etc. Also, it may be possible for the user to specify (set) which region is to be the region of interest by using a GUI while viewing the image.

[0033] That is, for example, the region of interest while driving is initially set to at least one of the driver's head 9, rear seat passenger's head 19, steering wheel 11, driver's hands 13, driver's feet 15, front window 21, etc. In other words, the region of interest while driving is the region including the driver's feet. It is desirable that the user be able to select and register at least one of these as the region of interest.

[0034] As shown by the diagonal lines in Fig. 2(B), the area of ​​interest when the vehicle is stopped is assumed to be the area outside the vehicle window, where there is almost no change in the vehicle environment while the vehicle is stopped, and where thieves or the like may approach the vehicle from the surroundings. Therefore, in this embodiment, by making it possible to see outside the windows arranged in a 360-degree direction, it is possible to photograph those who may harm the vehicle and monitor the surroundings. In other words, the area of ​​interest when the vehicle is stopped is the area including the outside of the vehicle window.

[0035] Specifically, these are the front window 21, the front seat side window 23, the rear seat side window 25, and the rear window 27, and since the outside of the vehicle through each window is the area of ​​interest, each piece of glass itself corresponds to the area of ​​interest from inside the vehicle.

[0036] However, because of the pillars (supports), the view is not completely 360 degrees, but in many cars, the glass area is larger than the pillars, and it is possible to observe outside the windows in all directions from inside the car, which can fully fulfill the function of photographing those who may be causing harm and monitoring the surroundings. Therefore, in this embodiment, the window area in all directions is set as the area of ​​interest when the car is stopped.

[0037] Furthermore, in the camera system according to this embodiment, by setting the optical characteristics and arrangement of the camera to predetermined conditions, it is possible to provide an inexpensive camera system that can effectively monitor the inside and outside of the vehicle using a single camera.

[0038] That is, in this embodiment, if the height of the interior space of the vehicle is relatively high, the viewing angle, which is the optical characteristic of the camera, is set to a wide angle of about 45 degrees, and the camera is placed facing downward on the ceiling of the vehicle interior, so that the above-mentioned areas of interest can be photographed simultaneously.

[0039] In addition, in the case of a vehicle in which the height of the interior space inside the vehicle cabin is relatively low, the positional relationship between the ceiling and the driver's head 9 is close to horizontal, so that when the optical axis of the camera is pointed downward, the viewing angle is set to about 90 degrees on one side so that the image can be captured almost horizontally.

[0040] In addition, the object of interest (area of ​​interest) while driving is mainly the face, while the object of interest (area of ​​interest) while stopped is mainly outside the surrounding windows, and the time period when the vehicle is moving and when stopped is different. Therefore, in the images captured by the camera, the face while driving and the windows while stopped are positioned so that they are almost in the same area. This allows the number of pixels of the image sensor to be allocated efficiently.

[0041] The camera position that satisfies these conditions is between the driver's seat and the passenger seat in the lateral direction of the vehicle, and between the driver's seat and the rear seat in the longitudinal direction of the vehicle, particularly in a position that simultaneously captures the driver's head 9 and the rear seat passenger's head 19. It is also desirable to place the camera in a position that simultaneously captures the front window 21, the front seat side window 23, the rear seat side window 25, and the rear window 27.

[0042] In addition, in this embodiment, as shown in Figures 2(A) and (B), in a fisheye camera with a viewing angle of approximately 90 degrees on one side, the camera is positioned so that the area of ​​interest is within a range from approximately 45 degrees to approximately 90 degrees.

[0043] That is, in this embodiment, the region of interest when the car is moving and the region of interest when the car is stopped are arranged so as to be within the range of φ / 2 to φ, where φ is the viewing angle of the wide-angle camera. Also, in this embodiment, the resolution in the range of φ / 2 to φ is configured to be higher than the resolution in the other ranges.

[0044] Therefore, subjects with relatively low importance as information are concentrated in the center. Also, since the distance from the ceiling to the subjects in the center is relatively short, the angle range that is occupied is narrow, and there is no need to increase the pixel density. In this embodiment, by using a camera with the optical characteristics described above, there is no need to use an image sensor with a high resolution.

[0045] A specific example of the image height characteristics of the camera in this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing the image height characteristics of a fisheye lens in the optical system according to the first embodiment, where the angle of view from the optical axis is from 0 degrees to 90 degrees.

[0046] In Fig. 3, 3a is an example of image height characteristics in the case of equidistant projection, characterized by a constant number of pixels per degree. In other words, when the coordinate y on the image sensor, the viewing angle (angle from the optical axis) φ, and k are constants, it can be expressed as y = k φ.

[0047] 3b is an example of image height characteristics in the case of stereoscopic projection, where the pixel density in the periphery is slightly higher than in the center, and can be expressed as y = 2k tan φ. 3c and 3d show examples in which the number of pixels per degree in the periphery is greater than in the stereoscopic projection of 3b.

[0048] In this embodiment, by using lenses with image height characteristics such as 3b, 3c, and 3d in the camera, the camera has a relatively high pixel density at 45 degrees to 90 degrees. Therefore, it is possible to image the region of interest with a high resolution suitable for the purpose without having to select an image sensor with an excessively large number of pixels.

[0049] Next, the camera system according to the first embodiment of the present invention will be described with reference to Fig. 3, Fig. 4(A), (B), and Fig. 5. Fig. 4(A) and (B) are diagrams showing an example of the relationship between the camera and the target of interest (region of interest) according to the first embodiment. Symbol ax in Fig. 4 is the distance in the front-rear direction between the wide-angle camera 3 and the driver's head 9, ay is the distance in the left-right direction (y direction) between the wide-angle camera 3 and the driver's head 9, and az is the distance in the height direction between the wide-angle camera 3 and the driver's head 9.

[0050] bx is the distance in the front-rear direction between the wide-angle camera 3 and the head 19 of the rear seat passenger, and bz is the distance in the height direction between the wide-angle camera 3 and the head 19 of the rear seat passenger. cx is the distance in the front-rear direction between the wide-angle camera 3 and the handlebar 11, and cz is the distance in the height direction between the wide-angle camera 3 and the handlebar 11.

[0051] dx indicates the front-to-rear distance between wide-angle camera 3 and driver's feet 15, and dz indicates the height direction distance between wide-angle camera 3 and driver's feet 15. ey is the left-right direction (y direction) distance from the camera to the bottom edge of front seat side window 23 and the bottom edge of rear seat side window 25. ez is the z direction distance from the camera to the bottom edge of front seat side window 23 and the bottom edge of rear seat side window 25.

[0052] Fig. 5 is a diagram showing an example of camera arrangement and an example of a determination result in embodiment 1. In Fig. 5, a, b, c, d, and e are arranged vertically, and x, y, and z are arranged horizontally as directions, and the values ​​of the cells where they intersect are the distances in the respective directions. ax to ez correspond to ax to ez in Fig. 4.

[0053] Incidentally, a to e correspond to the driver's head 9, the rear seat passenger's head 19, the steering wheel 11, the driver's feet 15, and the lower end of the front seat side window 23 (the lower end of the rear seat side window 25), respectively, as the objects of interest.

[0054] The angle θ is the angle between the horizontal and the line connecting the camera and each of the objects of interest corresponding to a to d. When the optical axis of the camera is set vertically downward (z direction), the angle φ between the camera optical axis and each line connecting the camera and each of the objects of interest corresponding to a to d is expressed as φ=90 degrees-θ.

[0055] The camera used in the first embodiment is one in which the number of pixels per degree increases sharply when the image height characteristic φ is 45 degrees or more, as shown in FIG. 3d. The image sensor is, for example, a 3M pixel image sensor. In that case, in order for the camera to satisfy the image height characteristic of FIG. 3d, the number of pixels per degree above 45 degrees is 13 pix. / deg.

[0056] However, in the range of 0 to 45 degrees, the number of pixels per degree is 7 pix. / deg. Also, overall, the average is 10 pix. / deg., and in the case of a fisheye camera with one side at 90 degrees from the optical axis, it is 10 x 90 = 900 pix (one side). Therefore, there are 900 x 2 = 1800 pixels in one direction, and since it is 1800 x 1800, an angular characteristic of this number of pixels per degree can be achieved with a 3M pixel image sensor.

[0057] Next, the camera arrangement in this embodiment using this camera will be described. In this embodiment, as shown in Fig. 5, the camera is arranged so that the objects of interest, such as the driver's head 9, the rear seat passenger's head 19, the steering wheel 11, and the driver's feet 15, are within a viewing angle range of 45 degrees to 90 degrees on one side from the optical axis of the camera.

[0058] Moreover, the distance in the y direction from the wide-angle camera 3 to the position of the driver's feet 15 is set to be the same as ay, which is the position of the center line of the body, because the position of the accelerator and brake is the position of greatest interest.

[0059] Also, the position where the angle φ is smallest in the window is the bottom end of the window where the distance between the camera and the x-direction is 0, so it is as shown by e in the table of Fig. 5, and this also satisfies the range of 45 to 90 degrees on one side from the optical axis of the camera. In this way, the camera can be positioned to satisfy the region of interest, and the region of interest while the vehicle is moving and the region of interest while the vehicle is stopped can be photographed with high resolution.

[0060] <Comparative Example 1> Next, a comparison with a camera system having an example of an arrangement different from that of the first embodiment of the present invention will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of a camera arrangement and an example of a determination result of a first comparative example which is different from that of the first embodiment. The cameras used in the first embodiment are the same, and only the arrangement of the cameras is different.

[0061] In the arrangement in Figure 6, the camera position in the x direction is shifted 20 cm forward of the car compared to the arrangement in Figure 5. Note that since it is considered that there is no problem in terms of crime prevention if more than half of the window is visible from the bottom edge of the camera, the top edge is indicated as f in the table, and the case where the height of ez is half is indicated as fz.

[0062] 6, the hands and feet are not within 45 to 90 degrees on one side of the optical axis of the camera, which is a high-resolution area, so it is not possible to capture the area of ​​interest at high resolution that meets the purpose. Also, if the camera is placed 10 cm behind the vehicle compared to embodiment 1, it will be placed exactly to the side of the driver, so the driver's facial expression will not be visible, and the driver monitoring system will not function, and the area of ​​interest cannot be observed.

[0063] <Comparative Example 2> Next, a comparison with a camera system having image height characteristics different from those of the first embodiment of the present invention will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of a comparative example 2 having image height characteristics different from those of the first embodiment. The position of the camera in this comparative example is almost the same as that of the first embodiment, and only the height characteristics of the camera are different.

[0064] 7, line 3d represents the image height characteristics of embodiment 1, and line 3e represents the image height characteristics of comparative example 2. In the case of the image height characteristics shown in 3e, the number of pixels per degree is 13 pix. / deg. even at angles of 45 degrees or more.

[0065] However, because the fisheye lens uses a lens with equidistant projection characteristics, even if one side is at an angle of 0 to 45 degrees from the optical axis of the camera, the number of pixels per side of the image sensor is 13 pix. / deg., and the number of pixels per side of the image sensor is 13 x 2 x 90 = 2340. Therefore, the total is 2340 x 2340 = approximately 5 M pixels, and a 5 M pixel image sensor that is more expensive than the 3 M pixel image sensor of the first embodiment is required.

[0066] <Embodiment 2> Next, a camera system having a control means (CPU, etc.) for obtaining a score for a region of interest according to embodiment 2 of the present invention will be described. In embodiment 2, the control means (CPU, etc.) can score the region of interest by indicating to what extent a specific position of the region of interest is included in the range of φ / 2 to φ of angle φ as a target viewing angle.

[0067] Alternatively, the proportion of the area of ​​interest that falls within the range of φ / 2 to φ with respect to the angle φ as the target viewing angle is scored. That is, the control means (CPU, etc.) calculates a score indicating the extent to which the pre-specified area of ​​interest is included within the range of φ / 2 to φ with the wide-angle camera installed. Alternatively, the area that is occupied may be scored. The control means is controlled by a computer executing a computer program.

[0068] In this embodiment, the score calculated based on the angle is displayed as a numerical value in real time. Also, the scores are displayed for the captured image and multiple points in the image. The higher the score, the more suitable it is for driving safely and observing for crime prevention while parked.

[0069] To identify the position, the head of the passenger or the mirror can be learned and identified by image recognition, and the area can be calculated by semantic segmentation. Alternatively, when adjusting the position of the camera, a pattern that makes it easier to recognize and identify, such as an AR marker, can be attached to the object of interest (area of ​​interest) to recognize it.

[0070] When adjusting the camera position, the angle values ​​for each target of interest (area of ​​interest) and the pass / fail result, as shown in Figure 5, are displayed on a tablet device in front of you, allowing you to adjust the camera mounting position and angle while looking at the pass / fail result on the device in front of you.

[0071] The camera position may be adjusted manually, or may be linked with an automatic adjustment device to automatically adjust the optimal placement angle by searching for the maximum score. That is, the above placement work is adjusted manually or automatically so that the area of ​​interest when the car is moving and the area of ​​interest when the car is stopped are placed within the range of φ / 2 to φ with respect to the angle φ as the viewing angle of the wide-angle camera.

[0072] <Embodiment 3> Next, Fig. 8 is a flowchart showing an example of a process of changing operating conditions during shooting in embodiment 3, and the operation of the camera system according to embodiment 1 of the present invention will be described with reference to Fig. 8. In embodiment 3, the operating conditions of the camera system are changeable.

[0073] That is, in the camera of the third embodiment, although the same area within the angle of view is used when the vehicle is moving and when it is stopped, the imaging conditions can be switched between when the vehicle is moving and when it is stopped, allowing the camera to be used more suitably.

[0074] The flowchart in Fig. 8 will now be described. Note that an in-camera CPU (not shown) serving as a computer provided inside wide-angle camera 3 executes a computer program stored in a memory (not shown) to sequentially perform the operations of the steps in the flowchart in Fig. 8.

[0075] When the camera power is turned on, the process of the flowchart in Fig. 8 starts. In step S81, the CPU in the camera determines the driving state and determines whether the car is stopped (driving). That is, if it is determined in step S81 that the car is not stopped, it is determined that the car is driving, and in step S82, shooting is performed under indoor imaging conditions.

[0076] That is, the accumulation time of the image sensor is controlled so that the signal level of the image obtained from the driver's head 9, the rear seat passenger's head 19, the steering wheel 11, the driver's feet 15, etc., which are the objects of interest, is optimized. Note that indoor white balance control may be performed. Also, focus adjustment is performed so that the contrast of the above objects of interest indoors, for example, the average value, is maximized.

[0077] Next, in step S83, the camera's internal CPU performs image recognition of indoor subjects using a DMS (Driver Monitoring System) based on the image data captured in step S81 to detect abnormalities (driver dozing, abnormal pulse, etc.).

[0078] If a predetermined abnormality is detected, a warning is displayed to the driver. In step S84, the indoor image and the recognition result in step S83 are stored in a storage medium (not shown), or are communicated to the outside and stored in an external server, etc. After that, the process proceeds to step S85.

[0079] On the other hand, if the answer in step S81 is Yes, that is, if it is determined that the vehicle is stopped, the process proceeds to step S85, where the camera CPU takes an outdoor image. That is, based on the brightness levels of the images of the outside of the vehicle through the front window 21 and rear window 27 as the subject of interest among the outputs from the image sensor, the accumulation time of the image sensor is controlled so that the brightness levels are optimized, and an image is taken.

[0080] In outdoor photography, the accumulation time is set to be shorter than that in indoor photography in step S82. In this case, white balance control may be performed according to the outdoor color temperature. In addition, focus adjustment is performed so that the contrast of the outdoor (outside the vehicle) target of interest is maximized, for example, by the average value.

[0081] Furthermore, when an obstacle is recognized by image recognition of the outdoor photographed result, the driving of the car may be controlled (by changing the direction or braking) to avoid the obstacle. Also, when a suspicious person is recognized outside the company while the car is stopped, a warning about the suspicious person may be displayed on the user's smartphone terminal or the like.

[0082] Furthermore, in step S86, the camera CPU stores the outdoor image and data related to the recognition results of obstacles, suspicious persons, etc. recognized in step S85 in a storage medium (not shown), or transmits the data to an external server, etc., and then proceeds to step S87.

[0083] In step S87, the camera CPU determines whether an operation to end the flow of Fig. 8 has been performed by the user, and if the answer is Yes, ends the flow of Fig. 8. If the answer is No in step S87, the process returns to step S81, and the flow of Fig. 8 is periodically repeated on a frame-by-frame basis.

[0084] In the flow of Fig. 8, an image for indoor use and an image for outdoor use are stored separately. However, for example, if step S81 is No, that is, while driving, an image optimized for low illuminance for indoor use and an image optimized for high illuminance for outdoor use may be synthesized to generate an image with a wide dynamic range and stored.

[0085] That is, in the camera of embodiment 3, the control means (CPU) adjusts the exposure or focus of the wide-angle camera to match the area of ​​interest outside the vehicle when the vehicle is stopped, and adjusts the exposure or focus of the wide-angle camera to match the area of ​​interest inside the vehicle when the vehicle is moving.

[0086] According to this embodiment, by performing the control shown in the flow of Fig. 8, it is possible to simultaneously realize face recognition as a driver monitoring function and a function as a front drive recorder with high resolution while driving. Note that while driving, indoor shooting in step S82 and outdoor shooting in step S85 are switched, for example, in units of a predetermined number of frames.

[0087] In addition, a specific region may be cut out, the dynamic range of the specific region may be limited, and the size may be reduced before saving. Furthermore, in order to obtain biological information such as the driver's drowsiness or inattention, it is necessary to subject the image information to recognition processing for the face region. In this case, the recognition processing is performed based on an image with the exposure and focus optimized for indoor use, so the image recognition accuracy can be improved.

[0088] Furthermore, if the vehicle is stopped and the answer in step S81 is Yes, exposure control and focus adjustment are optimized for the outdoor brightness and the distance of the outdoor subject, thereby improving the accuracy of recognizing suspicious persons outside the vehicle.

[0089] In addition, when the vehicle is stopped, the frame rate may be lowered to reduce the amount of data stored, since short-term image processing and judgment such as collision avoidance with an obstacle is not required as a shooting condition compared to when the vehicle is running outdoors. In the above description, the shooting conditions suitable for the vehicle running and the vehicle stopped are described, but the shooting conditions to be switched between the vehicle running and the vehicle stopped are not limited to the above-described examples.

[0090] <Embodiment 4> Next, a camera system according to a fourth embodiment of the present invention will be described. In the camera system according to the fourth embodiment, while the vehicle is stopped, image capturing is not performed at a constant cycle, but only when a proximity sensor using an infrared sensor or the like detects a moving object.

[0091] That is, the main purpose of taking pictures while the vehicle is stopped is crime prevention, and the target of interest is suspicious people approaching the vehicle, etc. Therefore, if a moving object outside the vehicle is not approaching the vehicle, there is little need to store the taken images.

[0092] Therefore, in the fourth embodiment, in consideration of the capacity of the memory unit for storing images, etc., when it is determined in step S81 that the vehicle is stopped and an object approaching the vehicle is detected by the proximity sensor, an image is captured in response to the detection, and the captured image and image recognition results are stored. That is, when the vehicle is stopped, an image is captured based on the output of another proximity sensor. This makes it possible to provide a suitable crime prevention system.

[0093] <Embodiment 5> Next, a camera system according to a fifth embodiment of the present invention will be described with reference to Fig. 9 and Fig. 10. Fig. 9(A) and (B) are diagrams for explaining an example of a region of interest in the camera system according to the fifth embodiment. Fig. 10 is a diagram showing an example of the relationship between the camera and an object of interest according to the fifth embodiment.

[0094] The fifth embodiment differs from the first embodiment in that it has a function of preventing small children from being left behind by treating the rear seats of a stopped vehicle as an area of ​​interest. That is, in the fifth embodiment, the area of ​​interest inside the vehicle when stopped includes the rear seats. Fig. 9(A) is a vertical cross-sectional view of the automobile, and Fig. 9(B) shows an image captured by the camera.

[0095] In the figure, 31 is a child left in the center of the back seat, and 32 is a child left at the edge of the back seat. Also, gx is the distance in the x direction from the wide-angle camera 3 to the child 31 left in the center of the back seat, and gz is the distance in the height direction from the wide-angle camera 3 to the child 31 left in the center of the back seat.

[0096] Other symbols are the same as in Fig. 1. In the fifth embodiment, as a function while the vehicle is stopped, in addition to a crime prevention function that monitors the outside of the vehicle, a function to prevent children from being left behind inside the vehicle is provided. For example, if a sleeping child is left behind in the back seat, the temperature inside the vehicle may rise, leading to dehydration and possibly a fatal accident.

[0097] To avoid this, a function to detect whether a child has been left behind in the vehicle is provided. That is, in the fifth embodiment, the region of interest is an area where a child may be left behind on the back seat.

[0098] Therefore, the camera position satisfies the relationship with other regions of interest, and furthermore, the gx, gz relationship with the camera is set as shown in Fig. 10. This allows the child 31 left in the center of the back seat to be photographed in a high-resolution region within a range of 45 to 90 degrees on one side from the optical axis of the camera.

[0099] Incidentally, FIG. 9(B) is a diagram showing an example of the positional relationship of objects of interest imaged through a lens, and not only child 31 left in the center of the back seat, but also child 33 left at the edge of the back seat can be imaged in a high-resolution area.

[0100] In addition, as a function to detect whether a child or other child has been left behind while the vehicle is stopped, while the conventional purpose of crime prevention is performed outside the vehicle, images are taken periodically (for example, about one frame per second) with exposure conditions and focus adjusted to the inside of the vehicle (indoors), and the abandoned child is detected and judged by image recognition. This makes it possible to achieve both the purpose of crime prevention and the purpose of abandoned child detection.

[0101] In addition to image recognition, the system may be configured to be able to visually determine whether or not something has been left behind when an inquiry is received from a user's mobile terminal, etc. For this purpose, when an inquiry is received from a user, an image may be taken by adjusting the exposure conditions and focus to the interior of the vehicle for a predetermined period of time, and the image obtained by this may be transferred to the user's mobile terminal and displayed. This allows the system to check whether or not something has been left behind from a remote location.

[0102] Although the present invention has been described in detail based on the preferred embodiment, the present invention is not limited to the above embodiment, and various modifications are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The above embodiment includes the following combinations.

[0103] (Configuration 1) A camera system comprising a wide-angle camera disposed within the passenger compartment of an automobile, the wide-angle camera being disposed between the driver's seat and the rear seat with its optical axis pointing downward, and the area of ​​interest when the automobile is moving and the area of ​​interest when the automobile is stopped being disposed within the range of φ / 2 to φ, with respect to angle φ being the viewing angle of the wide-angle camera.

[0104] (Configuration 2) The camera system according to configuration 1, wherein the wide-angle camera is a fisheye camera.

[0105] (Configuration 3) The camera system described in configuration 1 or 2, characterized in that the area of ​​interest when the vehicle is moving includes the heads of any persons inside the vehicle, and the area of ​​interest when the vehicle is stopped includes the area outside the vehicle windows.

[0106] (Configuration 4) The camera system according to any one of configurations 1 to 3, characterized in that the resolution in the range from φ / 2 to φ is higher than the resolution in the other ranges.

[0107] (Configuration 5) A camera system described in any one of configurations 1 to 4, characterized in that it has a control means for calculating, with the wide-angle camera installed, a score indicating to what extent the pre-specified region of interest is included in the range of φ / 2 to φ.

[0108] (Configuration 6) A camera system described in any one of configurations 1 to 5, characterized in that it has a control means for adjusting the exposure or focus of the wide-angle camera to match an area of ​​interest outside the vehicle when the vehicle is stopped, and for adjusting the exposure or focus of the wide-angle camera to match an area of ​​interest inside the vehicle when the vehicle is moving.

[0109] (Configuration 7) The camera system according to any one of configurations 1 to 6, characterized in that while the vehicle is stopped, photographing is performed based on an output from another proximity sensor.

[0110] (Configuration 8) The camera system according to any one of configurations 1 to 7, wherein the region of interest during driving includes the driver's feet.

[0111] (Configuration 9) The camera system according to any one of configurations 1 to 8, wherein the area of ​​interest inside the vehicle when stopped includes a back seat.

[0112] (Method) A method for installing a wide-angle camera to be placed inside the passenger compartment of an automobile, the wide-angle camera being placed between the driver's seat and the rear seat with its optical axis facing downward, and being positioned so that the area of ​​interest when the automobile is moving and the area of ​​interest when the automobile is stopped are within the range of φ / 2 to φ, with respect to angle φ as the viewing angle of the wide-angle camera, and the resolution in the range of φ / 2 to φ is higher than the resolution in other ranges.

[0113] (Program) A computer program for controlling the control means in the camera system according to configuration 5 or 6 by a computer.

[0114] In order to realize a part or all of the control in the above-mentioned embodiment, a computer program that realizes the functions of the above-mentioned embodiment may be supplied to a camera system or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the camera system or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0115] 1. Automotive 3. Wide-angle camera 5. Peripheral viewing angle region 6. Peripheral vision boundary 7. Driver 9...driver head 11. Handle 13. Driver's hands 15. Driver's feet 17 Rear seat passenger 19. Rear seat passenger head 21. Front window 23 Front seat side window 25 Rear seat side window 27 Rear window 31. Child left in the center of the back seat 33. Child left at end of rear seat

Claims

1. A wide-angle camera is disposed in the interior of the vehicle, The wide-angle camera is disposed between the driver's seat and the rear seat with its optical axis facing downward, The area of ​​interest when the vehicle is moving and the area of ​​interest when the vehicle is stopped are arranged so as to be within a range of φ / 2 to φ, with respect to the angle φ as the viewing angle of the wide-angle camera. A camera system comprising:

2. The camera system of claim 1 , wherein the wide-angle camera is a fish-eye camera.

3. the region of interest during driving includes a head of a person in the vehicle interior, The camera system of claim 1 , wherein the area of ​​interest when the vehicle is stopped includes outside a window of the vehicle.

4. 2. The camera system according to claim 1, wherein the resolution in the range of φ / 2 to φ is higher than the resolution in other ranges.

5. The camera system according to claim 1, further comprising a control means for calculating, with the wide-angle camera installed, a score indicating to what extent the pre-specified region of interest is included in the range of φ / 2 to φ.

6. The camera system according to claim 1, further comprising a control means for adjusting the exposure or focus of the wide-angle camera to match an area of ​​interest outside the vehicle when the vehicle is stopped, and for adjusting the exposure or focus of the wide-angle camera to match an area of ​​interest inside the vehicle when the vehicle is moving.

7. 2. The camera system according to claim 1, further comprising: a camera unit for capturing an image based on an output from another proximity sensor while the vehicle is stopped.

8. The camera system of claim 1 , wherein the region of interest during driving includes the driver's feet.

9. The camera system according to claim 1 , wherein the area of ​​interest inside the vehicle when stopped includes a rear seat.

10. A method for installing a wide-angle camera in a vehicle interior, comprising: The wide-angle camera is disposed between the driver's seat and the rear seat with its optical axis directed downward, The area of ​​interest when the vehicle is moving and the area of ​​interest when the vehicle is stopped are arranged to be within a range of φ / 2 to φ with respect to angle φ as the viewing angle of the wide-angle camera, and the resolution in the range of φ / 2 to φ is configured to be higher than the resolution in other ranges. An installation method characterized by:

11. 7. A computer program for controlling the control means in the camera system according to claim 5 or 6 by a computer.

Images