Mobile body and imaging device installation method

Abstract

To provide a movable body in which an imaging device is installed so as to be able to image a wide range with high resolution.SOLUTION: The movable body is configured such that an imaging device equipped with an optical system for forming, on the light reception surface of an imaging element, an optical image having a low-resolution region near an optical axis and a high-resolution region outside the low-resolution region is installed at an installation position for imaging at least one of a horizontal direction behind the side surface of one side of the movable body, one of left and right horizontal directions behind a driver's seat of the movable body, and a lower direction behind the movable body.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to a moving object such as an automobile, and a method for installing an imaging device. [Background technology]

[0002] In recent years, CMS (Camera Monitor System) technology, which replaces mirrors installed in automobiles with electronic mirrors, has been promoted. The imaging devices (cameras) installed for CMS are sometimes installed in multiple locations so that they can capture (imaging) images in all directions as the eyes of the electronic system to ensure safety. In addition, the United Nations regulation UN-R46 (regulations on indirect vision devices in motor vehicles) defines the blind spots of the driver, and CMS, like existing mirrors, are required to capture images and be installed in a way that reduces blind spots.

[0003] Furthermore, automobiles equipped with a CTA (Cross Traffic Alert) function are equipped with cameras capable of capturing images in the left and right directions at high resolution in the front and rear of the vehicle so as to detect the approach of other vehicles crossing in front or behind the vehicle. Furthermore, automobiles equipped with a front or rear monitoring device (front monitor or rear monitor) are equipped with wide-angle cameras capable of capturing images of a wide range in front or behind the vehicle. Patent Document 1 discloses that an image of the rear of a vehicle is captured by a camera equipped with an optical system capable of capturing an image with a high resolution in part of the imaging area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6349558 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the camera used in Patent Document 1, which has a high resolution in only part of its imaging area, is unable to capture high-resolution images over a wide area. Therefore, when it is necessary to capture high-resolution images over a wide area, it is necessary to prepare a large number of cameras so that they can cover that wide area.

[0006] Therefore, an object of the present invention is to provide a moving body in which an imaging device is arranged so as to be able to capture an image of a wide range with high resolution. [Means for solving the problem]

[0007] In order to achieve the above object, one aspect of the present invention is teeth, A mobile object, An imaging device having an optical system that forms an optical image on a light receiving surface of an imaging element, the optical image having a low resolution region in the vicinity of an optical axis and a high resolution region in a peripheral region of the low resolution region, In the low resolution region, In order to prevent the moving object from being imaged in an area corresponding to the optical axis center, and In the high resolution region, A horizontal direction behind one side of the moving body, and one of the left and right horizontal directions behind a driver's seat of the moving body; To capture like, Located behind the rear end of the moving body and outside the side of the moving body. It is characterized by being installed at the installation position. Effect of the Invention

[0008] According to the present invention, a moving body can be realized in which an imaging device is arranged so as to be able to capture an image of a wide range with high resolution. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram for explaining the positional relationship between a camera unit and a vehicle in the first embodiment. [Diagram 2] 1A and 1B are diagrams illustrating the optical characteristics of the optical system of the camera unit 11 in the first embodiment. [Diagram 3] FIG. 1 is a diagram for explaining an image processing system 100 according to a first embodiment. [Figure 4]1A and 1B are diagrams illustrating the relationship between the characteristics of the optical system of the camera unit 11 installed in the vehicle 1 of the first embodiment and the imaging area required for displaying the electronic side mirror and the forward CTA. [Diagram 5] 1 is a plan view of a vehicle 1 of a first embodiment seen from above. [Figure 6] FIG. 2 is a side view of the vehicle 1 of the first embodiment. [Figure 7] FIG. 2 is a front view of the vehicle 1 as seen from the front, for explaining the relationship between the camera unit 11 of the first embodiment and the vehicle 1. [Figure 8] 1 is a plan view of a vehicle 1 seen from above, illustrating a specified area under the front surface of the vehicle in the first embodiment. FIG. [Figure 9] 2 is a plan view of a vehicle 1 seen from above, illustrating a specified area at the rear of a side surface of the vehicle in the first embodiment. FIG. [Figure 10] 2 is a schematic plan view of a vehicle 1 seen from above to explain a specified area on a passenger seat side of the vehicle in the first embodiment. FIG. [Figure 11] FIG. 2 is a side view of the vehicle 1 of the first embodiment. [Figure 12] 13 is a diagram for explaining the positional relationship between a camera unit and a vehicle in the second embodiment. FIG. [Figure 13] 11 is a plan view of a vehicle 1 seen from above, illustrating a specified area under the front surface of a vehicle in a second embodiment. FIG. [Figure 14] 13 is a diagram for explaining a specified area behind a vehicle in the second embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions are omitted or simplified.

[0011] [Embodiment 1] In the first embodiment, a method for installing a camera unit that captures high-resolution images for electronic side mirrors and images for forward CTA, and also captures images for a forward monitoring device, will be described. Fig. 1 is a diagram for explaining the positional relationship between the camera unit and the vehicle in the first embodiment.

[0012] In the example shown in Fig. 1, a camera unit 11 as a first imaging device is installed on the front left side of a vehicle 1 which is an automobile, and a camera unit 12 is installed on the front right side. Note that, in this embodiment, it is described that two camera units are installed on the vehicle 1, but the number of camera units is not limited to two, and it is sufficient if there is at least one camera unit. Also, it is sufficient if multiple camera units capable of capturing images in the required imaging range and high-resolution imaging direction are installed.

[0013] The camera units 11 and 12 are installed so as to capture the rear side of the vehicle 1 to capture images for the electronic side mirror, either the left or right front side of the vehicle 1 to capture images for the forward CTA, and a forward lower image for the forward monitoring device of the vehicle 1.

[0014] In this embodiment, the optical systems of camera units 11 and 12 generate an image circle having a higher resolution at the peripheral angle of view compared to the center in the optical axis direction, and form an image on an image element described later. In this embodiment, the optical characteristics of camera units 11 and 12 are assumed to be approximately the same, and the characteristics will be exemplarily described using FIG. 2.

[0015] 2A and 2B are diagrams for explaining the optical characteristics of the optical system of camera unit 11 in this embodiment. Fig. 2A is a diagram showing the image circle of the optical system of camera unit 11 and the image height y at each half angle of view on the light receiving surface of the image sensor in the form of contour lines.

[0016] Fig. 2(B) is a diagram showing the projection characteristics of an image circle that represents the relationship between the image height y and the half angle of view θ of the optical system of camera unit 11. In Fig. 2(B), the half angle of view (the angle between the optical axis and the incident light) θ is plotted on the horizontal axis, and the image height (image height) y on the light receiving surface (image plane) of the imaging element of camera unit 11 is plotted on the vertical axis.

[0017] As shown in Fig. 2B, in this embodiment, the projection characteristic y(θ) is configured to be different between an area less than a predetermined half angle of view θa and an area equal to or greater than the half angle of view θa. If the increase in image height y per unit half angle of view θ, that is, the index expressed as the differential value dy(θ) / dθ of the projection characteristic y(θ) at the half angle of view θ, is taken as the resolution, it can be said that the greater the slope of the projection characteristic y(θ) in Fig. 2B, the higher the resolution.

[0018] In addition, it can be said that the resolution is higher as the interval between the image heights y at each half angle of view of the contour lines in Fig. 2A becomes larger. In this way, the optical system of the camera unit 11 has optical characteristics in which the resolution is higher in the region where the half angle of view is equal to or larger than the half angle of view θa than in the region where the half angle of view is less than the half angle of view θa.

[0019] It is assumed that the projection characteristic y(θ) of the optical system shown in FIGS. 0.2< 2*ftan(θmax / 2) / y(θmax) <0.92 (Math. 1)

[0020] In this embodiment, the peripheral region of the light receiving surface where the half angle of view θ is equal to or larger than θa is called the high resolution region 10a, and the central region of the light receiving surface where the half angle of view θ is less than θa is called the low resolution region 10b. That is, the optical system of this embodiment has projection characteristics in which the image height per unit angle of view is higher in the high resolution region than in the low resolution region near the optical axis, and has a low resolution region near the optical axis and a high resolution region outside the low resolution region.

[0021] In this embodiment, the circle of the boundary between the high resolution region 10a and the low resolution region 10b is called the resolution boundary, and the boundary image on the display screen corresponding to the resolution boundary is called the display resolution boundary or simply the boundary image. However, the boundary between the high resolution region 10a and the low resolution region 10b is not limited to a circle, and may be an ellipse with different projection characteristics y(θ) for each radial direction, or a distorted shape.

[0022] In this embodiment, a lens having the above characteristics is called a stereoscopic projection lens, and a camera unit equipped with a stereoscopic projection lens is called a stereoscopic projection camera. In this embodiment, the center of gravity of the low-resolution region 10b of the light receiving surface of the image sensor of the camera unit and the center of gravity of the high-resolution region 10a are almost overlapped, and the center of gravity of the high-resolution region 10a is almost coincident with the position where the optical axis of the optical system of the camera unit intersects with the light receiving surface of the image sensor.

[0023] However, for example, the center of gravity of the light receiving surface of the imaging element and the center of gravity of the high resolution area 10a may be shifted from each other in a specified direction, or the center of gravity of the high resolution area 10a may be shifted from the position where the optical axis of the optical system intersects with the light receiving surface.

[0024] Next, the configuration of the image processing system in this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram for explaining the image processing system 100 in the first embodiment, and the image processing system 100 in Fig. 3 is mounted on a vehicle 1. The camera unit 11 is made up of an imaging section 21 and a camera processing section 31, and the camera unit 12 is made up of an imaging section 22 and a camera processing section 32. The imaging sections 21 and 22 are made up of stereoscopic projection lenses 21a and 22a, and imaging elements 21b and 22b such as CMOS image sensors, respectively.

[0025] The stereoscopic projection lenses 21a and 22a, which are the optical systems of the camera units 11 and 12, are each composed of one or more optical lenses, and each has the optical characteristics shown in Fig. 2(A) and (B). The stereoscopic projection lenses 21a and 22a form (focus) their respective optical images on the light receiving surfaces of the image sensors 21b and 22b, respectively. The image sensors 21b and 22b function as imaging means, and perform photoelectric conversion on the focused optical images to output imaging signals. For example, R, G, and B color filters are arranged for each pixel in a Bayer array on the light receiving surfaces of the image sensors 21b and 22b, and R, G, and B pixel signals are sequentially output from the imaging unit 21 as imaging signals.

[0026] The camera processing units 31 and 32 have a function of processing the imaging signals output from the imaging units 21 and 22, respectively. The camera processing units 31 and 32 each have an image processing unit 31a and 32a, a recognition unit 31b and 32b, and a camera information unit 31c and 32c, respectively.

[0027] The camera processing units 31 and 32 each include a built-in CPU as a computer and a memory as a storage medium that stores a computer program, and the CPU executes the computer program in the memory, thereby executing the processing in the camera processing units 31 and 32.

[0028] Incidentally, the camera processing units 31 and 32 are not limited to the above configuration, and the image processing units 31a and 32a and the recognition units 31b and 32b may be configured with hardware such as a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP).

[0029] The image processing units 31a and 32a perform various image correction processes such as white balance adjustment, gain / offset adjustment, gamma processing, color matrix processing, lossless compression processing, and distortion correction processing on the imaging signals output from the imaging units 21 and 22. The image processing units 31a and 32a are configured to perform distortion correction on the image signals of the low resolution region 10b but not on the image signals of the high resolution region 10a.

[0030] Also, for example, image data input from the imaging units 21 and 22 according to the Bayer array is de-Bayer processed and converted into image data in RGB raster format. Note that part of the image processing as described above may be performed by other blocks of the image processing system 100 other than the image processing units 31a and 32a.

[0031] The recognition units 31b and 32b have a function of performing image recognition of a predetermined object (e.g., a car, a person, an obstacle, etc.) from the image signals image-processed by the image processing units 31a and 32a, respectively, and outputting a first image recognition result to the integrated processing unit 40. For example, the image recognition result is transmitted to the integrated processing unit 40 as a set of the type and coordinates of the object.

[0032] In addition, the recognition units 31b and 32b may receive prediction information including information on the type of object and the moving direction of the object or priority recognition area information from the integrated control unit 41c of the integrated processing unit 40, and perform image recognition processing.

[0033] The camera information units 31c and 32c function as storage means for storing camera information such as characteristic information relating to the characteristics of optical images and position and orientation information of the camera units, and store in advance in memory the camera information of the camera units 11 and 12. The camera information units 31c and 32c may also store information from various sensors provided in the camera units 11 and 12.

[0034] The camera information to be held includes, for example, characteristic information of the stereoscopic projection lenses 21a and 22a, the number of pixels of the imaging elements 21b and 22b, mounting position information and attitude (pitch, roll, yaw, etc.) of the camera units 11 and 12 in the vehicle coordinate system, optical axis direction, imaging range, etc. The camera information may also include information such as gamma characteristics, sensitivity characteristics, frame rate, image format, etc. of the video output from the camera processing unit 31.

[0035] The mounting position information of the camera units is stored in advance in a memory in the camera information section as relative coordinates with respect to the vehicle 1, since the mounting position of each camera unit with respect to the vehicle is often determined for each camera unit. The camera information may also be information specific to the imaging sections 21, 22 (for example, lens aberration characteristics or image sensor noise characteristics). The camera information is transmitted to the integrated processing section 40, and is referred to when the integrated processing section 40 performs image processing necessary for displaying the image on the display section.

[0036] The integrated processing unit 40 has a function of displaying the video signals obtained from the camera units 11 and 12 on display devices such as the first display unit 50 and the second display unit 51, and a function of notifying the warning display unit 52 and the audio notification unit 53. Furthermore, the integrated processing unit 40 has a function of image recognition of an object that straddles the boundary between the imaging ranges of the camera units 11 and 12, etc.

[0037] The integrated processing unit 40 has a SOC (System On Chip) / FPGA (Field Programmable Gate Array) 41, a CPU 42 as a computer, and a memory 43 as a storage medium. In this embodiment, the integrated processing unit 40 will be described as being housed in a housing separate from the camera unit.

[0038] Incidentally, some or all of the functional blocks included in the integrated processing unit 40 and the like may be realized by hardware, or may be realized by the CPU 42. As the hardware, a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP) or the like may be used. The CPU 42 performs various controls of the entire image processing system 100 by executing a computer program stored in the memory 43.

[0039] The SOC / FPGA 41 has an image processing unit 41a, a recognition unit 41b, and an integrated control unit 41c. The image processing unit 41a has a function of performing image processing such as resolution conversion for displaying on a display device based on the image signals and camera information acquired from the camera processing units 31 and 32.

[0040] As described above, the camera information includes the optical characteristics of the stereoscopic projection lenses 21a, 22a, the number of pixels of the image pickup elements 21b, 22b, photoelectric conversion characteristics, gamma characteristics, sensitivity characteristics, format information of the image signal, coordinates of the mounting position of the camera unit in the vehicle coordinate system, and attitude information.

[0041] For example, the image processing unit 41a smoothly combines the image signals of the low-resolution region 10b of each of the imaging units 21 and 22, which have been subjected to distortion correction, with the image signals of the high-resolution region 10a to form an entire image for each of the imaging units 21 and 22. The image processing unit 41a also performs image processing such as image rotation based on the camera arrangement position and attitude information in particular of the camera information, and transmits the image signals to the display device.

[0042] The recognition unit 41b performs image recognition processing on a composite image formed by stitching together images from the imaging units 21 and 22, and performs image recognition of a specified object (e.g., a car, a person, an obstacle, etc.) in the composite image obtained from the imaging units 21 and 22 to generate a second image recognition result.

[0043] At that time, the recognition results (type and coordinates of the object) by the recognition units 31b and 32b may also be referred to. In the above, the recognition unit 41b performs image recognition on the composite image obtained from the imaging units 21 and 22, but it is not necessarily required to perform image recognition on the composite image.

[0044] The explanation will continue assuming that the recognition unit 41b is capable of outputting a second image recognition result as a result of image recognition of an image signal of an area larger than the partial area on which image recognition was performed by the first image recognition means, among the image signals acquired by the image acquisition means.

[0045] The integrated control unit 41c functions as an integrated processing means for outputting an integrated image recognition result based on the reliability of the first image recognition result and the reliability of the second image recognition result. For example, when the recognition result by the recognition units 31b and 32b differs from the recognition result by the recognition unit 41b, the integrated control unit 41c outputs the integrated image recognition result by adopting the recognition result with the higher reliability.

[0046] For example, the proportion of an object in an image recognized by recognition units 31b and 32b is compared with the proportion of the same object within the screen recognized by recognition unit 41b, and the recognition result with the larger proportion is determined to be more reliable and is adopted.

[0047] The integrated control unit 41c also has a function of forming a video signal for displaying a desired image on the first display unit 50, the second display unit 51, etc., from the entire image of each of the image-processed imaging units 21, 22. In addition, the integrated control unit 41c generates a frame for highlighting a recognized object, and CG for information and warnings regarding the type, size, position, speed, etc., of the object.

[0048] Furthermore, CG of a boundary image for displaying the boundary may be generated based on characteristic information of the optical system, such as display resolution boundary information acquired from the camera information units 31c and 32c. Then, display processing is performed for superimposing the CG and characters on the image. Here, the first display unit 50, the second display unit 51, etc. function as display means, and display the image signal and the integrated image recognition result.

[0049] Moreover, the integrated control unit 41c communicates with the driving control unit (ECU) 60 and the like via a communication means (not shown) provided inside using a protocol such as CAN, FlexRay, or Ethernet. This allows the integrated control unit 41c to perform a display process for appropriately changing the information to be displayed based on the vehicle control signal received from the driving control unit (ECU) 60 and the like. That is, for example, the range of an image to be displayed on the display means can be changed according to the moving state of the vehicle acquired by the vehicle control signal.

[0050] The driving control unit (ECU) 60 is mounted on the vehicle 1 and is a unit incorporating a computer and memory for comprehensively performing drive control, direction control, etc. of the vehicle 1. The driving control unit (ECU) 60 inputs information related to the driving (moving state) of the vehicle, such as the driving speed, driving direction, shift lever, shift gear, and blinker state, the vehicle orientation based on a geomagnetic sensor, etc., and GPS information, as vehicle control signals to the integrated processing unit 40.

[0051] The integrated control unit 41c may also have a function of transmitting information such as the type, position, moving direction, and moving speed of a predetermined object (such as an obstacle) recognized by the recognition unit 41b to the driving control unit (ECU) 60. As a result, the driving control unit (ECU) 60 performs control required for obstacle avoidance, such as stopping the vehicle, driving, and changing the driving direction. The driving control unit (ECU) 60 functions as a movement control means for controlling the movement of the vehicle based on the integrated image recognition results.

[0052] The first display unit 50 is a display device that is installed, for example, near the center in the width direction of the vehicle 1 at the upper front part of the driver's seat with the display screen facing the rear of the vehicle and functions as an electronic rearview mirror. The first display unit 50 may be configured to be a half mirror or the like and to be usable as a mirror when not used as a display.

[0053] The first display unit 50 may also be configured to have a touch panel or operation buttons to receive instructions from a user and output them to the integrated control unit 41c. The first display unit 50 may also be used as an electronic side mirror for checking obstacles on the left and right, or a display device that displays an image for a forward CTA, instead of a conventional optical side mirror. The first display unit 50 receives and displays an image signal with a field of view necessary for display according to the purpose of the first display unit, generated by the integrated control unit 41c.

[0054] The second display unit 51 is a display device that is installed, for example, around an operation panel near the center of the vehicle width direction in front of the driver's seat of the vehicle 1, and functions, for example, as a display device for displaying the downward direction in front of the vehicle. Like the first display unit, the second display unit 51 also functions as a display device that receives a necessary imaging area generated according to the purpose. The integrated control unit 41c receives and displays a signal required for display according to the purpose of the second display unit.

[0055] For example, the second display unit can display various control signals from a navigation system, an audio system, and the driving control unit (ECU) 60. Also, the second display unit may be provided with a touch panel and operation buttons, and may be configured to be able to receive instructions from the user.

[0056] Also, the second display unit 51 may be, for example, a display unit of a tablet terminal, and can display by being connected to the integrated processing unit 40 by wire, or can receive and display an image wirelessly via the communication unit 62. Note that as the display elements of the first display unit 50 and the second display unit 51, a liquid crystal display, an organic EL display, or the like can be used, and the number of display units is not limited to one.

[0057] The integrated control unit 41c determines whether a moving object is included in the image based on the recognition results output by the recognition units 31b, 32b, and 41b, and outputs the recognition result. The moving object here is, for example, a bicycle, a pedestrian, or another vehicle, and is called a detection target in this embodiment. The recognition result output by the integrated control unit 41c includes the presence or absence of a detection target, the type and coordinates of the detection target, and speed information.

[0058] The warning display unit 52 functions as a warning display means, and issues, for example, a side collision warning to the driver using visual information based on the recognition result output from the integrated control unit 41c. The warning display unit 52 may be configured, for example, with an LED, and configured to light up or flash when the recognition result includes information that a detection target exists.

[0059] The warning display unit 52 may be configured with a display such as a liquid crystal display. In this case, when the recognition result includes information that a detection target exists, an icon or text information is output on the display. The warning display unit 52 may be installed, for example, near the end of the vehicle width direction in front of the driver's seat of the vehicle 1 with the display screen facing the driver.

[0060] Moreover, the warning display unit 52 may be installed near the first display unit 50 and the second display unit 51, or may be configured to substitute for the first display unit 50 and the second display unit 51. The audio notification unit 53 functions as an audio notification means, and outputs a sound based on the recognition result output from the integrated control unit 41c. For example, a sound can be output using a speaker to notify the driver. It is desirable to install the audio notification unit 53 near the end of the vehicle 1 in front of the driver's seat in the vehicle width direction, for example.

[0061] The integrated control unit 41c also functions as a warning condition determination means, and controls the warning display unit 52 or the voice notification unit 53 to change the content of the warning output based on the coordinates and speed of the detection target obtained by the recognition units 31b to 32b and the recognition unit 41b. The integrated control unit 41c may also be configured to control the warning display unit 52 to raise the level of warning as the distance to the detection target becomes closer.

[0062] For example, it is desirable to control the audio notification unit 53 so that the volume of the notification increases as the distance between the detection target and the side of the vehicle decreases. The integrated control unit 41c may be configured to determine whether the traveling speed of the vehicle input from the traveling control unit (ECU) 60 is equal to or lower than a predetermined value, and to display a warning on the warning display unit 52 or to give a warning notice on the audio notification unit 53 only when the traveling speed is equal to or lower than the predetermined value.

[0063] When the vehicle is traveling at a high speed, there is a possibility that the detection target may not be recognized correctly, so by using this configuration, the possibility of performing correct side detection can be improved. In this case, the predetermined value for the vehicle's traveling speed is preferably set to 30 km / h.

[0064] Furthermore, when the detection target is moving, there is a higher possibility of collision with the vehicle compared to when the detection target is stationary. Therefore, the integrated control unit 41c may be configured to determine whether the speed of the detection target is within a predetermined range, and to display a warning in the warning display unit 52 or to notify a warning in the audio notification unit 53 only when it is determined that the speed of the detection target is within the predetermined range. In this case, it is preferable that the predetermined range of the speed of the detection target is 5 km / h or more and 20 km / h or less.

[0065] In addition, the integrated control unit 41c determines whether the vehicle is turning right or left based on the vehicle movement direction information output from the driving control unit (ECU) 60, and displays a warning in the warning display unit 52 or issues a warning notification in the audio notification unit 53 only when the vehicle is turning right or left.

[0066] In this embodiment, the integrated processing unit 40 is mounted on the vehicle 1, but some of the processing of the image processing unit 41a, the recognition unit 41b, and the integrated control unit 41c of the integrated processing unit 40 may be performed by an external server, for example, via a network.

[0067] In this case, for example, the imaging units 21-24 as image acquisition means are mounted on the vehicle 1, but for example, some of the functions of the camera processing units 31, 32 and the integrated processing unit 40 can be processed by an external server or the like. Also, it is possible to have some or all of the functions of the integrated processing unit 40 in the driving control unit (ECU) 60.

[0068] Reference numeral 61 denotes a storage unit, which records the entire images generated by the integrated processing unit 40 for each of the imaging units 21 and 22. Furthermore, it records CG such as predetermined frames, characters, and warnings indicating recognized objects, and images superimposed with CG and displayed on the first display unit 50, the second display unit 51, etc., together with the time, GPS information, etc. The integrated processing unit 40 can also reproduce past information recorded in the storage unit 61 and display it on the first display unit 50 or the second display unit 51.

[0069] Reference numeral 62 denotes a communication unit for communicating with an external server or the like via a network, and is capable of transmitting information before being recorded in the memory unit 61 and past driving history information recorded in the memory unit 61 to an external server or the like and storing the information in the external server or the like.

[0070] As described above, the image can be transmitted to an external tablet terminal or the like and displayed on the second display unit 51, which is the display unit of the tablet terminal. It is also possible to obtain traffic congestion information and various other information from an external server and display it on the first display unit 50 and the second display unit 51 via the integrated processing unit 40. Reference numeral 63 denotes an operation unit for inputting various instructions to the image processing system through user operations. The operation unit includes, for example, a touch panel and operation buttons.

[0071] Next, the relationship between the characteristics and the arrangement of the camera unit 11 installed on the left front side and the camera unit 12 installed on the right front side will be described. Figures 4(A) and (B) are diagrams for explaining the relationship between the characteristics of the optical system of the camera unit 11 installed on the vehicle 1 of the first embodiment and the imaging area required for displaying the electronic side mirror and the forward CTA. Figure 4(A) is a side view of the vehicle 1 as seen from the side, and Figure 4(B) is a front view of the vehicle as seen from the front.

[0072] The X, Y, and Z directions in Fig. 4 have the installation position of the camera unit 11 as the origin, and the X direction is the horizontal direction toward the front of the vehicle as shown in Fig. 4(A). The Z direction is the horizontal direction to the left as you face the vehicle 1 as shown in Fig. 4(B). The Y direction is the vertical upward direction as shown in Figs. 4(A) and (B).

[0073] In the XY plane in Fig. 4(A), the angle is positive toward the X direction and negative toward the -X direction with respect to the vertical direction. In the YZ plane in Fig. 4(B), the angle is positive toward the Z direction and negative toward the -Z direction with respect to the vertical direction. E in the XY plane in Fig. 4(A) represents the driver's viewpoint position, which is the position of the driver's eyes when seated or the center position of the driver's seat surface.

[0074] Using the above definitions, the relationship between the optical axis of the camera unit 11 and the imaging angle of view will be described. φv, θv, and θlv on the XY plane in FIG. 4(A) will be described. φv is the optical axis direction of the camera unit 11 and is the angle from the vertical direction. In this embodiment, the optical axis of the imaging device is directed downward from the installation position. θv is the angle of view of the high resolution area 10a of the camera unit 11. θlv is the angle of view of the low resolution area 10b of the camera unit 11.

[0075] φh, θh, and θlh in the YZ plane in Fig. 4(B) will be described. φh is the optical axis direction of the camera unit 11 and is the angle from the vertical direction. θh is the angle of view of the high resolution area 10a of the camera unit 11. θlh is the angle of view of the low resolution area 10b of the camera unit 11.

[0076] Using the above definitions, we will explain the positional relationship of camera unit 11. In the XY plane in Fig. 4(A), x is the horizontal distance between camera unit 11 and the driver's viewpoint position E. h1 is the vertical distance between camera unit 11 and the ground.

[0077] 4(B), z in the YZ plane is the distance between the installation position (first installation position) of the camera unit 11 and the side of the vehicle 1. Here, the imaging area (prescribed area behind the side of the vehicle) required for electronic side mirror display will be described with reference to FIGS.

[0078] Fig. 5 is a plan view of the vehicle 1 of the first embodiment seen from above, and the X and Z directions correspond to Fig. 4 with the origin being the installation position of the camera unit 11. The specified area behind the side of the vehicle is an area on the ground as shown by the hatching in Fig. 4(A) and Fig. 5, and is an area that the driver needs to check using the side mirror of the vehicle.

[0079] The specified region behind the side of the vehicle is a region on the ground surface defined by a width w1 from the side of the vehicle body at a distance d1 behind the driver's viewpoint position E, and a width w2 from the side of the vehicle body at a distance d2, with the passenger side region being region 1 and the driver side region being region 2. Region 1 and region 2 are shaped symmetrically with respect to the center line of the vehicle 1 in the longitudinal direction.

[0080] The following describes the installation conditions for the camera unit 11 installed on the passenger seat side to be able to capture high-resolution images in the horizontal direction while capturing images of a specified region 1 behind the side of the vehicle. The distant end of region 1, which is the specified region behind the side of the vehicle, continues to infinity behind the vehicle, so the camera unit 11 is installed so that it can capture images at infinity, that is, in the horizontal direction of width w2. Also, in order to confirm distant objects far from the driver using images captured by the camera, it is desirable to install the camera so that it can capture images in the nearby ground direction to the horizontal direction with high resolution.

[0081] Therefore, in embodiment 1, the camera unit 11 is installed to satisfy the following installation condition (Equation 2) so that the horizontal angle (-90°) is included within the range of the high resolution area 10a (φv-θv-θlv) and (angle of φv-θlv). φv-θv-θlv<-90°<φv-θlv (Math. 2)

[0082] In addition, in order to include the specified region region 1 of the side of the vehicle in the imaging angle of view of the camera unit 11, in addition to the above equation (2), the camera unit is installed so that the tip region (region with width w1) in the direction of travel of the specified region region 1, located at a distance d1 rearward from the viewpoint position E, is included in the imaging angle of view.

[0083] In other words, the camera is installed so that the angle (-Atan((x+d1) / h)) in the direction of the tip region of region 1, which is d1 away from the viewpoint position E, is included in the angle of view of camera unit 11 (φv-θv-θlv) to (φv+θv+θlv), satisfying the following condition (equation 3).

[0084] φv-θv-θlv≦-Atan((x+d1) / h1)≦φv+θv+θlv····(Quantity 3) By installing the camera unit 11 so as to satisfy the installation conditions (first condition) of the above-mentioned formulas 2 and 3, the camera unit 11 can capture high-resolution images in the horizontal direction while capturing an image of a specified region 1 behind one side surface of the vehicle 1. In this case, the one side surface of the vehicle 1 is the passenger seat side surface.

[0085] Next, the installation conditions for the camera unit 11 to capture images in the horizontal direction with high resolution for displaying the forward CTA will be described. For displaying the forward CTA, it is desirable to display an object approaching from a distance on the left and right at an intersection with poor visibility. Therefore, in the first embodiment, the camera unit 11 is installed in front of the driver's viewpoint position E so that the left and right horizontal directions can be appropriately confirmed.

[0086] That is, since the left and right horizontal directions continue to infinity, the camera unit is installed so that it can capture nearby objects in the horizontal direction with high resolution. Here, an example of the installation condition (Equation 4, Equation 5) as the second condition for camera unit 11 to be able to capture at least one of the left and right horizontal directions forward of the driver's viewpoint position E (driver's seat) will be described.

[0087] In the first embodiment, when the camera unit 11 is located in front of the viewpoint position E (x≧0), it is installed so as to satisfy the following installation condition (second condition) of equation 4 or equation 5. That is, at least one of the high resolution areas 10a in the left-right direction ((φh-θh-θlh) to (φh-θlh) on the right side when facing the front of the vehicle, and (φh+θlh) to (φh+θh+θlh) on the left side when facing the front of the vehicle) includes the left-right horizontal direction.

[0088] φh-θh-θlh≦-90°≦φh-θlh (Equation 4) φh+θlh≦90°≦φh+θh+θlh (Equation 5)

[0089] When camera unit 11 is located behind viewpoint position E (x<0), it is desirable to install it so that the high-resolution area includes the horizontal area in front of viewpoint position E. By installing it in this way, camera unit 11 can capture an image of either side of the vehicle in the horizontal direction with high resolution.

[0090] In order to capture images of the left and right directions in front of the vehicle and the rear of the side of the vehicle, it is desirable to place the camera unit 11 at a position where the vehicle 1 is not captured at the center of the optical axis. For example, in order to capture images of the left and right directions in front of the vehicle, it is desirable that the offset x of the camera unit is larger than the distance d0 from the viewpoint position E to the front end of the vehicle 1. Furthermore, in order to capture images of the rear of the side of the vehicle, it is desirable to install the offset z of the camera unit 11 on the outside of the side of the vehicle 1 (in the -Z direction in FIG. 4).

[0091] As described above, by satisfying the first condition of the formulas 2 and 3 and the second condition of the formula 4 or 5, it is possible to include either the rear side or the left or right direction of the vehicle 1 within the imaging angle of view. Furthermore, in order to include the front lower part of the vehicle within the imaging angle of view, in addition to the above conditions of the formulas 2 and 3 and the formula 4 or 5, it is desirable to point the optical axis toward the ground.

[0092] The imaging area (prescribed area under the front of the vehicle) required for the display of the lower front used for the forward monitoring device will be described with reference to Fig. 6 to Fig. 8. Fig. 6 is a side view of the vehicle 1 of the first embodiment, Fig. 7 is a front view of the vehicle 1 from the front for explaining the relationship of the camera unit 11 of the first embodiment with the vehicle 1, and Fig. 8 is a plan view of the vehicle 1 from above for explaining the prescribed area under the front of the vehicle of the first embodiment. The X, Y, and Z directions and angles in Fig. 6 to Fig. 8 correspond to Fig. 4(A), (B), and Fig. 5, respectively, and have the same relationship.

[0093] E in Figures 6 and 8 indicates the driver's viewpoint position, and the positional relationship (x, h1, z) of the camera unit 11 with respect to the vehicle 1 and the relationship between the optical axis and the imaging angle of view (Φv, θv, θlv, Φh, θh, θlh) in Figures 6 and 7 are the same as those in Figures 4(A) and (B). Here, the specified area below the front of the vehicle will be explained with reference to Figures 6 to 8.

[0094] The specified area under the front of the vehicle is the area that the driver needs to check using the side under mirror of the vehicle. Here, d7 is the distance from the front end of the vehicle 1 to the far area under the front that needs to be checked, d0 is the distance from the viewpoint E to the front end of the vehicle 1, w6 is the width of the vehicle 1, and w7 is the distance to the far area on the side of the vehicle that needs to be checked.

[0095] That is, the specified area below the front surface of the vehicle is the area on the ground surrounded by d7 and (w6+w7). Therefore, the installation condition for camera unit 11 to capture an image of the specified area below and in front of vehicle 1 is the condition (third condition) that the imaging angle of view of camera unit 11 includes the area surrounded by d7 and (w6+w7).

[0096] Therefore, in the first embodiment, the camera unit 11 is installed so as to satisfy the installation conditions (Equation 6, Equation 7) as the third condition, so that the angle of view of the front end of d7 is included within the imaging angle of view of (φv-θv-θlv) to (φv+θv+θlv).

[0097] φv-θv-θlv≦-Atan((x-d0) / h1)≦φv+θv+θlv (Math. 6) φv-θv-θlv≦Atan((d0+d7-x) / h1)≦φv+θv+θlv (Math. 7)

[0098] Also, the camera unit 11 is installed so that the imaging angle of view of the camera unit 11 includes the ends of w6 and w7 within the range of (w6+w7) and satisfies the installation conditions (Equation 8, Equation 9). φh-θh-θlh≦-Atan((w6+z) / h1)≦φh+θh+θlh (Math. 8) φh-θh-θlh≦Atan((w7-z) / h1)≦φh+θh+θlh (Math. 9)

[0099] In this embodiment, by installing the imaging device in this manner, the camera unit 11 can realize a moving body capable of capturing images of a wide range of a predetermined specified area below the front of the vehicle with high resolution. Note that this embodiment does not need to satisfy all of the first, second, and third conditions, and includes an installation that satisfies at least one of the first to third conditions.

[0100] In the case of large vehicles, the field of view is made wider than the specified area at the rear of the side of the vehicle (specified wide area at the rear of the side of the vehicle). The specified wide area at the rear of the side of the vehicle will be described with reference to Fig. 9. Fig. 9 is a plan view of the vehicle 1 viewed from above, for explaining the specified area at the rear of the side of the vehicle in the first embodiment. The areas on the ground to the sides and rear of the vehicle shown by diagonal lines in Fig. 9 are the same as the specified area at the rear of the side of the vehicle described in Fig. 5.

[0101] The specified region of the wide rear area of ​​the side of the vehicle is a region (region 3, region 4) on the ground surface defined by a width w3 from the side of the vehicle body at a distance d3 rearward from the driver's viewpoint position E, and a width w4 from the side of the vehicle body at a distance d4 rearward from the viewpoint position E. The camera unit 11 arranged on the passenger seat side can be installed so as to include the specified region 3 of the wide rear area of ​​the side of the vehicle in its imaging range, thereby meeting the installation conditions of a large vehicle.

[0102] In particular, in large vehicles, it is necessary to capture an image of a detection area for detecting moving objects on the side of the passenger seat of the vehicle and for side collision warning to warn the driver if there is a possibility of a collision. This detection area is an area where there is a high possibility of entrapment or collision if an object is present within the detection area when the vehicle is turning left, for example. The camera unit 11 arranged on the passenger seat side can be installed so that the detection area is included in the imaging range, thereby meeting the installation conditions for detecting moving objects on the passenger seat side.

[0103] The relationship between the specified area on the passenger side of the vehicle, the detection area, and an object will be described with reference to Figs. 10 and 11. Fig. 10 is a schematic plan view of the vehicle 1 seen from above to explain the specified area on the passenger side of the vehicle in embodiment 1. Fig. 11 is a side view of the vehicle 1 in embodiment 1 seen from the side. The X, Y, and Z directions and angles in Figs. 10 and 11 have the same relationships as those in Figs. 6 to 8. E in Figs. 10 and 11 is the same as E in Figs. 6 to 8, and represents the driver's viewpoint position.

[0104] The specified area on the passenger side of the vehicle is an area surrounded by the distance d8 from the front of the vehicle to the rear of the vehicle where detection is required, and the distance w8 from the side of the vehicle to the far side of the vehicle where detection is required, as shown in Figure 10. The size of the moving object that needs to be detected is an object with a height of h2. Therefore, the detection area for side collision warning is an area surrounded by the specified area on the passenger side of the vehicle and the height h2.

[0105] The camera unit 11 can be installed so that the detection area for the side collision warning is included in the angle of view, thereby meeting the installation requirements for large vehicles. It is desirable that the height h1 of the camera unit 11 is higher than the height h2 of the moving object to be detected. In other words, it is desirable to satisfy the condition of the following formula 10. h1 ≧ h2 (Equation 10)

[0106] By setting these installation conditions, it is possible to include a moving object of height h2 in the imaging range without making the imaging angle of the camera unit wider than when h1≦h2 is satisfied. For example, if bicycles are to be detected as a detection target, it is desirable to set h2 to 1.7 meters. Also, if pedestrians are to be detected as a detection target, it is desirable to set h2 to 2 meters. If the offset x of the camera unit from the front end of the vehicle is large, the entire ground surface of an area of ​​width w8 at that position must be included in the imaging range in order to capture a detection target that is within the detection area.

[0107] The above describes the installation conditions for camera unit 11. In order to capture high-resolution images of multiple required imaging ranges and high-resolution imaging directions with as few cameras as possible, it is desirable to place camera unit 11 diagonally outward and forward from the corner of the vehicle body on the passenger seat side.

[0108] The installation conditions for the camera unit 11 on the passenger seat side have been described above. However, by installing the camera unit 12 on the driver's seat side under the same installation conditions, it is possible to capture high-resolution images of multiple required imaging ranges and high-resolution imaging directions with fewer cameras. It is preferable to install the camera unit 12 diagonally outward from the vehicle body corner on the driver's seat side.

[0109] The camera unit 11 installed in front of the passenger seat will be described below with actual specific values. For example, the case of a passenger car with 9 or less occupants or a small cargo vehicle (for example, with a vehicle weight of 3.5 tons or less) and with d0=0.8 m will be described.

[0110] In UN-R46, the specified area behind the side of the vehicle is defined as d1 = 4m, d2 = 20m, w1 = 1m, and w2 = 4m. In addition, in UN-R46, the specified area under the front of the vehicle is defined as d7 = 0.3m, w6 = 1.8m, and w7 = 0.3m. In addition, in the safety standards set by the Ministry of Land, Infrastructure, Transport and Tourism (Article 21 (Driver's Seat) Detailed Notification Article 27 and Attachment 29 Ground Area of ​​Safety Standards for Road Transport Vehicles), d7 = 2.3m, d8 = 4.8m, and w8 = 0.3m.

[0111] It is desirable that the driver can confirm these specified areas visually or through a mirror or a display device. A specific example of the case where the ground area is included in the imaging area of ​​the camera unit 11 of this embodiment will be described.

[0112] For example, assume that a camera unit with a vertical angle of view θv and a horizontal angle of view θh of 30.0 degrees, and a vertical angle of view θlv and a horizontal angle of view θlh of 60 degrees in the high resolution region 10a is installed at a position of x=0.8 m, z=0.3 m, and h=1.2 m. In this case, the vertical angle φv and the horizontal angle φh of the optical axis of the camera unit may be set to -28.56°<φv<0.0° and -30.0°<φh<30.0°, respectively, which can be calculated from the above-mentioned equations 2 to 9.

[0113] Also, for example, in the case of a large freight vehicle with a total vehicle weight of 8.0 tons or more, it is desirable to be able to confirm a larger number of areas. Therefore, it is desirable for the driver to be able to visually or by using a mirror or a display device confirm the specified areas of d0 = 1.0 m, d1 = 4 m, d2 = 30 m, d3 = 1.5 m, d4 = 10 m, d5 = 1 m, d6 = 1.75 m, d7 = 2 m, d8 = 12 m, w1 = 1 m, w2 = 5 m, w3 = 4.5 m, w4 = 15 m, w5 = 2 m, w6 = 2 m, w7 = 2 m, and w8 = 3 m, which are similarly obtained.

[0114] A specific example of the case where the camera unit 11 of this embodiment includes this region in the imaging region will be described. For example, a camera unit is used in which the vertical angle of view θv and the horizontal angle of view θh of the high resolution region 10a are 23.6 degrees, and the vertical angle of view θlv and the horizontal angle of view θlh are 66.4 degrees. The camera unit is installed at a position of x=1.2 m, z=0.3 m, and h=2.3 m such that the vertical angle φv of the optical axis of the camera unit satisfies -23.6°<φv<0.0°, which can be obtained from the above-mentioned equations 2, 3, 6, 7, and 10.

[0115] Furthermore, since it is desirable to include the horizontal direction to the left in the angle of view in order to capture wide-angle images of the rear side, it is desirable to set the angle φh of the horizontal optical axis of the camera unit to -23.6°<φh<0°, which can be obtained from the above-mentioned equations 4, 8, 9, and 10.

[0116] In the above numerical example, the installation position (x, z, h) of the camera unit is set as a fixed condition, and the angle condition of the optical axis of the camera unit is calculated in that case, but the method of calculating the installation condition is not limited to this as long as it satisfies the conditions of equations 2 and 3. For example, if the angle of the optical axis of the camera unit is predetermined as a constraint on the vehicle design, the range of the installation position of the camera unit that satisfies the conditions of equations 2 and 3 is determined based on that.

[0117] This allows the specified area to be included in the imaging range of the high resolution area of ​​the camera unit. Similarly, when the installation position (x, z, h) of the camera unit and the angle of the optical axis of the camera unit are predetermined, the vertical angle of view θv and the horizontal angle of view θh of the high resolution area 10a of the camera unit that satisfy the conditions of Equations 2 and 3 may be determined based on them. As described above, it is possible to image multiple specified areas with one camera, and further to image two or more horizontal directions with high resolution.

[0118] [Embodiment 2] Fig. 12 is a diagram for explaining the positional relationship between the camera unit and the vehicle in the second embodiment, and in the second embodiment, a camera unit having optical characteristics as shown in Fig. 2 is installed at the rear corner of the vehicle 1 as shown in Fig. 12. In the second embodiment, a camera installation method is explained that can capture multiple required imaging ranges and high-resolution imaging directions when capturing images of the rear side with fewer cameras than the number of directions required for high-resolution imaging. Note that the explanation of the parts that have already been explained in the first embodiment will be omitted.

[0119] First, the installation conditions will be described below for camera unit 11' as a second imaging device installed at the rear corner of vehicle 1 to capture an image of a specified region 1 behind the side of the vehicle in the same manner as in embodiment 1. The optical system and image sensor characteristics of camera unit 11' described in embodiment 1 are approximately the same as those of camera unit 11.

[0120] That is, the second imaging device has an optical system that forms an optical image on the light receiving surface of the imaging element, the optical image having a low resolution area near the optical axis and a high resolution area outside the low resolution area. Although the offset x between the camera unit 11 and the driver's viewpoint position E is different, the same installation conditions can be used.

[0121] That is, since the distant area of ​​the specified area behind the side of the vehicle continues to infinity behind the vehicle, the camera unit is installed so that it can capture images of infinity, i.e., the horizontal direction of width w2. Also, in order to confirm distant objects far from the driver using images captured by the camera, it is desirable to install the camera so that it can capture images of the nearby ground in the horizontal direction with high resolution.

[0122] Also, in order for the imaging angle of view of camera unit 11' to include the specified region region1 of the vehicle side, in addition to equation 2, an area w1 that is d1 away from viewpoint position E is included in the imaging angle of view. Therefore, as an example of the installation, it is desirable to have an installation that satisfies condition 1 (equation 2, equation 3). That is, by capturing an image of the horizontal direction behind one side of vehicle 1 in the high-resolution region of the optical system, camera unit 11' can capture an image of the horizontal direction at high resolution while capturing an image of specified region region1 of the side of vehicle 1 in FIG. 5.

[0123] The following describes in detail the installation conditions for camera unit 11' to capture images in the horizontal direction with high resolution for the rear CTA display. The rear CTA display requires a display that allows the driver to check objects approaching from far away on the left and right at intersections with poor visibility, and camera unit 11' must be installed so that it can check one of the horizontal directions, left and right, behind the driver's viewpoint position E (driver's seat).

[0124] Since the left and right horizontal directions continue to infinity, it is desirable to install the camera unit so that it can capture nearby objects in the horizontal direction with high resolution. An example of the installation conditions (Equation 4 and Equation 5) that allow camera unit 11' to capture at least one of the left and right horizontal directions behind the driver's viewpoint position E will be described below.

[0125] Consider the case where camera unit 11' is located behind viewpoint position E (x<0). In that case, the camera unit is installed so that at least one of high-resolution areas 10a in the left and right directions ((φh-θh-θlh) to (φh-θlh) on the right side when facing the front of the vehicle, and (φh+θlh) to (φh+θh+θlh) on the left side when facing the front of the vehicle) includes the left and right horizontal directions.

[0126] Furthermore, when camera unit 11' is located forward of viewpoint position E (x≧0), it is installed so that the horizontal area behind viewpoint position E is included in the high resolution area. By installing it in this manner, camera unit 11' can capture an image of either side of the vehicle in the horizontal direction with high resolution.

[0127] In order to capture images of the left and right directions behind the vehicle rear and the rear of the vehicle side, it is desirable to place camera unit 11' at a position where vehicle 1 is not captured at the center of the optical axis. For example, in order to capture images of the left and right directions behind the vehicle rear, it is desirable that the offset x of the camera unit is larger than the front end of the rear of vehicle 1. Furthermore, in order to capture images of the rear of the vehicle side, it is desirable to install camera unit 11' with offset z on the outer side of vehicle 1 (-Z direction in FIG. 4).

[0128] As described above, by satisfying the conditions of the formulas 2 and 3 and the formula 4 or 5, either the rear side or the left or right direction of the vehicle 1 can be included within the imaging angle of view. Furthermore, in addition to the first condition of the formulas 2 and 3 and the second condition of the formula 4 or 5, it is desirable to satisfy the following fourth condition in order to include the downward direction behind the vehicle 1 within the imaging angle of view. However, this embodiment may satisfy any one of the first condition, the second condition, and the fourth condition.

[0129] The imaging area (prescribed area under the rear surface of the vehicle) required for the rearward-downward display used for the rear monitoring device will be described with reference to Fig. 13. Fig. 13 is a plan view of the vehicle 1 seen from above, for explaining the prescribed area under the front surface of the vehicle in the second embodiment. The X, Y, and Z directions and angles in Fig. 13 correspond to and have the same relationship as Figs. 4(A), (B), and 5, respectively. E in Fig. 13 is the same as E in Figs. 4(A) and 5, and represents the driver's viewpoint position.

[0130] The positional relationship (x, h1, z) of the camera unit 11' with respect to the vehicle 1 and the relationship between the optical axis and the imaging angle of view (Φv, θv, θlv, Φh, θh, θlh) shown in Fig. 13 are the same as those in Fig. 4(A) and (B). d8 in Fig. 13 is the length of the vehicle 1 (vehicle length).

[0131] Here, the specified area under the rear of the vehicle will be described with reference to Fig. 13. The specified area under the rear of the vehicle is an area that the driver needs to check on the display device when backing up the vehicle, and the distance from a predetermined position on the near side to be checked, which is a distance d10 away from the rear of the vehicle, to a predetermined position on the far side to be checked is d11, and the width of the vehicle 1 is w6.

[0132] In this case, the specified area below the rear of the vehicle is the area on the ground surrounded by d11 and w6, and it is desirable that the camera unit 11' be able to capture an object of height h3 within the specified area to be confirmed behind the vehicle 1.

[0133] Therefore, the installation condition (fourth condition) under which camera unit 11' can capture an image of a specified area below the rear surface of the vehicle is a condition under which the imaging angle of view of camera unit 11' includes an area surrounded by d11, w6, and height h3.

[0134] That is, the camera unit 11' is installed so that the above-mentioned area is included within the imaging angle of view (φv-θv-θlv) to (φv+θv+θlv) and the following installation conditions (Equations 11, 12, and 14) or (Equations 11, 13, and 15) are satisfied. Note that z is the distance between the installation position (second installation position) of the camera unit 11' and the side of the vehicle.

[0135] φv-θv-θlv≦-Atan((d8-d0-x+d10) / h1)≦φv+θv+θlv (Math. 11)

[0136] (When h1 ≧ h3) φv-θv-θlv≦-Atan((d8-d0-x+d11) / (h1-h3))≦φv+θv+θlv (Math. 12)

[0137] (h1 <h3のとき) φv-θv-θlv≦-Atan((h3-h1) / (d8-d0-x+d11))-90°≦φv+θv+θlv...(Math. 13)

[0138] (When h1 ≧ h3) φh-θh-θlh≦Atan((w6+z) / (h1-h3))≦φh+θh+θlh (Math. 14)

[0139] (h1 <h3のとき) φh-θh-θlh≦Atan((h3-h1) / (w6+z))+90°≦φh+θh+θlh (Math. 15)

[0140] By installing the camera unit 11' in this manner, the camera unit 11' can properly capture an image of a specified area below and behind the vehicle. In particular, in a large vehicle, it is necessary to capture an image of a detection area for a side collision warning that detects a moving object on the side of the passenger seat of the vehicle and warns the driver in the event of a possible collision.

[0141] This detection area is an area where there is a high possibility of entrapment or collision if an object is present within the detection area when the vehicle is turning left, for example. The camera unit 11 disposed on the passenger side can be installed so that the detection area is included in the imaging range, thereby meeting the installation conditions for detecting a moving object on the passenger side.

[0142] The relationship between the specified area on the passenger side of the vehicle, the detection area, and the object is the same as in Figures 10 and 11. Here, the range of width w5 and distance d5 shown in Figures 6 and 8 under the passenger side of the vehicle is defined as the specified area, and installation conditions under which camera unit 11' can capture an image will be described.

[0143] For example, in addition to equations 8 and 10, it is desirable to satisfy the following installation condition (equation 16) so that the above region is included within the imaging angle of view of camera unit 11' (φv-θv-θlv to φv+θv+θlv).

[0144] v-θv-θlv≦Atan((x+d5) / h1)≦φv+θv+θlv (Math. 16)

[0145] In addition, since an image of the distant area behind the vehicle needs to be displayed on the display unit so that the driver can check the area behind the vehicle using the electronic rearview mirror, it is desirable to capture an image of a specified area behind the vehicle in a high-resolution area so that the driver can check the distant area.

[0146] Fig. 14 is a diagram for explaining a specified area behind the vehicle in the second embodiment, and as shown in Fig. 14, the specified area behind the vehicle is a range of width w12 and distance d12. Also, the width from the side of the vehicle to the specified area is w11. The installation conditions under which the camera unit 11' can capture an image will be described.

[0147] To capture an image of this specified area, it is desirable to satisfy the installation conditions (Equations 17 and 18) so that the above area is included within the imaging angle of view of camera unit 11' (φv-θv-θlv) to (φv+θv+θlv).

[0148] φv-θv-θlv≦-Atan((d12-x) / h1)≦φv-θlv (Math. 17) φh-θh-θlh≦Atan((w12-w11-z) / h1)≦φh+θh+θlh (Math. 18)

[0149] The above describes the installation conditions for camera unit 11'. In order to capture multiple required imaging ranges and high-resolution imaging directions with fewer cameras than the number of directions required for high-resolution imaging, it is desirable to place camera unit 11' on the outer side diagonally rearward from the rear corner on the passenger seat side.

[0150] The installation conditions for the camera unit 11' on the passenger seat side have been described above. By arranging the camera unit 12' on the driver's seat side under the same installation conditions, it is possible to capture multiple required imaging ranges and high-resolution imaging directions on the driver's seat side with fewer cameras than the number of directions required for high-resolution imaging. It is also desirable to arrange the camera unit 12' on the outside diagonally rearward from the rear corner on the driver's seat side.

[0151] The following describes actual specific values ​​for the camera unit 11' on the passenger seat side. For example, in the case of a passenger car with 9 or less occupants or a small cargo vehicle (for example, vehicle weight 3.5 tons or less), it is desirable for the driver to be able to check the following specified areas with the electronic side mirror: d0=0.9m, d1=4, d2=20m, d8=4.8m, d10=0.3m, d11=3.5m, d12=60m, w1=1m, w2=4m, w5=0.3m, w6=1.8m, w11=9m, w12=20m.

[0152] In this case, for example, if a camera unit is used in which the vertical angle of view θv and the horizontal angle of view θh of the high resolution area 10a are 23.6 degrees, and the vertical angle of view θlv and the horizontal angle of view θlh are 66.4 degrees, and the camera unit is installed at a position x = 3 m, z = 0.3 m, and h = 1.4 m, the vertical angle φv and the horizontal angle φh of the optical axis of the camera unit will be -22.2°<φv<0° and -7.06°<φh<9.14°, respectively, as calculated by the above-mentioned equations 2 to 5 and 11 to 18.

[0153] In addition, for example, in the case of a large cargo vehicle with a total vehicle weight of 8.0 tons or more, it is desirable to be able to check more areas, so it is desirable for the driver to be able to check the following specified areas using the electronic side mirror: d0=1m, d1=4m, d3=1.5m, d4=10m, d5=1m, d6=1.75m, d7=2m, d8=12m, d10=0.3m, d11=3.5m, d12=60m, w1=1m, w2=5m, w3=4.5m, w4=15m, w5=2m, w8=3m, w6=2m, w11=9m, w12=20m.

[0154] In this case, assume that a camera unit is used in which the vertical and horizontal angles of view θv and θh of the high resolution region 10a are 23.6 degrees, and the vertical and horizontal angles of view θlv and θlh of the high resolution region 10a are 66.4 degrees, and the camera unit is installed at a position of x = 12 m, z = 0.2 m, and h = 2 m. Then, the vertical angle φv of the optical axis of the camera unit is -8.75°<φv<0°, which can be calculated using the above-mentioned equations 2, 3, 10, 11, 12, 13, 16, and 17.

[0155] Furthermore, since it is desirable to include the horizontal direction to the left in the angle of view in order to capture wide-angle images of the rear side, the angle φh of the horizontal optical axis of the camera unit is −9.87°<φh<0°, which can be calculated using the above-mentioned equations 4, 8, 10, 14, 15, and 18.

[0156] In the above numerical example, the installation position (x, z, h) of the camera unit is set as a fixed condition, and the angle condition of the optical axis of the camera unit is calculated in that case, but the method of calculating the installation condition is not limited to this as long as it satisfies the conditions of equations 2 and 3. For example, if the angle of the optical axis of the camera unit is predetermined as a constraint on the vehicle design, the range of the installation position of the camera unit that satisfies the conditions of equations 2 and 3 is determined based on that.

[0157] This allows the specified area to be included in the imaging range of the high resolution area of ​​the camera unit. Similarly, when the installation position (x, z, h) of the camera unit and the angle of the optical axis of the camera unit are predetermined, the vertical angle of view θv and the horizontal angle of view θh of the high resolution area 10a of the camera unit that satisfy the conditions of Equation 2 and Equation 3 may be determined based on them.

[0158] As described above, in this embodiment as well, a single camera can capture images of multiple specified areas, and can capture images of two or more horizontal directions with high resolution. Note that, in this embodiment, the camera unit having the optical characteristics shown in Fig. 2 is disposed at the second installation position at the rear, but it may be combined with a camera unit disposed at the front as in the first embodiment.

[0159] That is, for example, when the vehicle 1 is viewed from above, a total of four camera units may be arranged, two at both front corners and two at both rear corners. Alternatively, a total of two camera units may be installed, one at the front corner, for example, on the passenger seat side, and one at the rear corner diagonally opposite the driver's seat side. Alternatively, two may be arranged at the front corner and one at the rear corner, or one at the front corner and two at the rear corners.

[0160] In addition, at least a part of the information processing system of the embodiment may or may not be mounted on a moving object such as a vehicle. The present invention can also be applied to a case where a moving object is remotely controlled.

[0161] Although the present invention has been described in detail based on the preferred embodiment, the present invention is not limited to the above embodiment, and various modifications are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. Note that this embodiment includes the following combinations.

[0162] (Configuration 1) A moving body, characterized in that an imaging device having an optical system that forms an optical image on the light receiving surface of an imaging element having a low resolution area near an optical axis and a high resolution area outside the low resolution area is installed at an installation position where the high resolution area of ​​the optical system images at least one of the following: a horizontal direction behind one side of the moving body, one of the left and right horizontal directions behind the driver's seat of the moving body, and a downward direction behind the moving body.

[0163] (Configuration 2) A moving body according to configuration 1, characterized in that the optical system has projection characteristics in which the image height per unit angle of view is higher in the angle of view of the high resolution region than in the angle of view of the low resolution region near the optical axis.

[0164] (Configuration 3) The moving body according to configuration 1 or 2, characterized in that the optical axis direction of the optical system is downward from the installation position.

[0165] (Configuration 4) When the image height of the optical system is y and θ is a half angle of view, the projection characteristic y(θ) satisfies the following condition: 0.2< 2*ftan(θmax / 2) / y(θmax) <0.92 4. The moving body according to any one of configurations 1 to 3, wherein the above is satisfied.

[0166] (Configuration 5) The moving body according to any one of configurations 1 to 4, wherein the one side surface of the moving body is a passenger seat side surface of the moving body.

[0167] (Configuration 6) When the moving body is viewed from the side, the angle of the optical axis direction from the vertical direction is φv, the angle of view of the high resolution area is θv, the angle of view of the low resolution area is θlv, a predetermined distance behind the driver's viewpoint is d1, the distance between the viewpoint position and the installation position is x, and the height of the installation position is h1. φv-θv-θlv<-90°<φv-θlv φv-θv-θlv≦-Atan((x+d1) / h1)≦φv+θv+θlv 6. The moving body according to any one of configurations 1 to 5, wherein the above is satisfied.

[0168] (Configuration 7) When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the front is φh, the angle of view of the high resolution area is θh, and the angle of view of the low resolution area is θlh, φh-θh-θlh≦-90°≦φh-θlh φh+θlh≦90°≦φh+θh+θlh 7. The moving body according to any one of configurations 1 to 6, wherein the above is satisfied.

[0169] (Configuration 8) Regarding the imaging device, when the moving body is viewed from the side, the angle of the optical axis direction from the vertical is φv, the angle of view of the high resolution area is θv, the angle of view of the low resolution area is θlv, the distance between the driver's viewpoint position and the installation position is x, the height of the installation position is h1, the distance from the viewpoint position to the tip of the moving body is d0, the length of the moving body is d8, the distance from the moving body to a predetermined position on the near side to be confirmed rearward is d10, the distance from the moving body to a predetermined position on the far side to be confirmed rearward is d11, the width of the moving body is w6, the height of an object to be confirmed rearward of the moving body is h3, and the distance between the installation position and the side of the moving body is z. φv-θv-θlv≦-Atan((d8-d0-x+d10) / h1)≦φv+θv+θlv (When h1 ≧ h3) φv-θv-θlv≦-Atan((d8-d0-x+d11) / (h1-h3))≦φv+θv+θlv (h1 <h3のとき) φv-θv-θlv≦-Atan((h3-h1) / (d8-d0-x+d11))-90°≦φv+θv+θlv (When h1 ≧ h3) φh-θh-θlh≦Atan((w6+z) / (h1-h3))≦φh+θh+θlh (h1 <h3のとき) φh-θh-θlh≦Atan((h3-h1) / (w6+z))+90°≦φh+θh+θlh 8. The moving body according to any one of configurations 1 to 7, wherein the above is satisfied.

[0170] (Configuration 9) An imaging device having an optical system that has a low-resolution area in the vicinity of an optical axis and forms an optical image having a high-resolution area outside the low-resolution area on a light receiving surface of an imaging element, The high-resolution region of the optical system is installed at a position that satisfies at least one of the following first, second, and third conditions, The first condition is: When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side is φv, the angle of view of the high resolution area is θv, the angle of view of the low resolution area is θlv, a predetermined distance behind the driver's viewpoint is d1, the distance between the viewpoint and the installation position is x, and the height of the installation position is h1. φv-θv-θlv<-90°<φv-θlv φv-θv-θlv≦-Atan((x+d1) / h1)≦φv+θv+θlv The condition is satisfied. The second condition is: When the moving body is viewed from the front, the angle of the optical axis direction from the vertical direction is φh, the angle of view of the high resolution area is θh, and the angle of view of the low resolution area is θlh, φh-θh-θlh≦-90°≦φh-θlh φh+θlh≦90°≦φh+θh+θlh The condition is satisfied. The third condition is: When the distance from the viewpoint position to the tip of the moving body is d0, the length of the moving body is d8, the distance from the moving body to a predetermined position on the near side to be confirmed rearward is d10, the distance from the moving body to a predetermined position on the far side to be confirmed rearward is d11, the width of the moving body is w6, the height of an object to be confirmed rearward of the moving body is h3, and the distance between the installation position and the side of the moving body is z, φv-θv-θlv≦-Atan((d8-d0-x+d10) / h1)≦φv+θv+θlv (When h1 ≧ h3) φv-θv-θlv≦-Atan((d8-d0-x+d11) / (h1-h3))≦φv+θv+θlv (h1 <h3のとき) φv-θv-θlv≦-Atan((h3-h1) / (d8-d0-x+d11))-90°≦φv+θv+θlv (When h1 ≧ h3) φh-θh-θlh≦Atan((w6+z) / (h1-h3))≦φh+θh+θlh (h1 <h3のとき) φh-θh-θlh≦Atan((h3-h1) / (w6+z))+90°≦φh+θh+θlh A moving object characterized in that the above conditions are satisfied.

[0171] (Method) An installation method in which an imaging device equipped with an optical system that forms an optical image on the light receiving surface of an imaging element having a low resolution area near the optical axis and a high resolution area outside the low resolution area is installed in an installation position where the high resolution area of ​​the optical system images at least one of the following: a horizontal direction behind one side of a moving body, one of the horizontal directions to the left and right behind the driver's seat of the moving body, and a downward direction behind the moving body.

[0172] In order to realize a part or all of the control in this embodiment, a computer program that realizes the functions of the above-mentioned embodiment may be supplied to an imaging device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the imaging device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0173] 1: Vehicle 11: Camera unit 12: Camera unit 100: Image processing system 21: Imaging unit 22: Imaging unit 31: Camera processing unit 32: Camera processing unit 40: Integrated processing unit 41: SOC / FPGA 42: CPU 43: Memory 50: 1st display section 51:Second display section 52: Warning display section 53: Voice notification section 60: Driving control ECU 61: Storage part 62: Communications Department 63:Operation unit

Claims

1. A moving object, an imaging device having an optical system for forming an optical image on a light receiving surface of an imaging element, the optical image having a low resolution region in the vicinity of an optical axis and a high resolution region in a peripheral region of the low resolution region; In the low-resolution area, the moving object is not imaged in an area corresponding to the optical axis center, and in the high resolution region, an image is captured in a horizontal direction behind one side of the moving body and in one of the horizontal directions to the left and right behind a driver's seat of the moving body. A movable body, characterized in that it is installed at an installation position rearward of a tip end of a rear surface of the movable body and outward of a side of the movable body.

2. 2 . The moving body according to claim 1 , wherein the optical system has projection characteristics in which an image height per unit angle of view is higher in the high resolution region than in the low resolution region near the optical axis.

3. 2. The moving body according to claim 1, wherein an optical axis direction of the optical system is a downward direction from the installation position.

4. When the image height of the optical system is y and θ is a half angle of view, the projection characteristic y(θ) satisfies the following condition: 0.2<2*ftan(θmax / 2) / y(θmax)<0.92 2. The moving body according to claim 1, wherein the above formula satisfies the above formula.

5. 2. The vehicle according to claim 1, wherein the one side surface of the vehicle is a passenger seat side surface of the vehicle.

6. the optical system is configured such that a projection characteristic corresponding to the low resolution region is different from a projection characteristic corresponding to the high resolution region; When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side is φv, the angle of view of the high resolution area is θv, the angle of view of the low resolution area is θlv, a predetermined distance behind the driver's viewpoint is d1, the distance between the viewpoint position and the installation position is x, and the height of the installation position is h1, φv-θv-θlv<-90°<φv-θlv φv-θv-θlv≦-Atan((x+d1) / h1)≦φv+θv+θlv 2. The moving body according to claim 1, wherein the above formula satisfies the above formula.

7. the optical system is configured such that a projection characteristic corresponding to the low resolution region is different from a projection characteristic corresponding to the high resolution region; When the moving body is viewed from the front, the angle of the optical axis direction from the vertical direction is φh, the angle of view of the high resolution area is θh, and the angle of view of the low resolution area is θlh, φh-θh-θlh≦-90°≦φh-θlh φh+θlh≦90°≦φh+θh+θlh 2. The moving body according to claim 1, wherein the above formula satisfies the above formula.

8. the optical system is configured such that a projection characteristic corresponding to the low resolution region is different from a projection characteristic corresponding to the high resolution region; Regarding the imaging device, when the moving body is viewed from the side, the angle of the optical axis direction from the vertical direction is φv, the angle of view of the high resolution area is θv, the angle of view of the low resolution area is θlv, the distance between the driver's viewpoint position and the installation position is x, the height of the installation position is h1, the distance from the viewpoint position to the tip of the moving body is d0, the length of the moving body is d8, the distance from the moving body to a predetermined position on the near side to be confirmed rearward is d10, the distance from the moving body to a predetermined position on the far side to be confirmed rearward is d11, the width of the moving body is w6, the height of an object to be confirmed rearward of the moving body is h3, and the distance between the installation position and the side of the moving body is z, φv-θv-θlv≦-Atan((d8-d0-x+d10) / h1)≦φv+θv+θlv (When h1 ≧ h3) φv-θv-θlv≦-Atan((d8-d0-x+d11) / (h1-h3))≦φv+θv+θlv (When h1<h3) φv-θv-θlv≦-Atan((h3-h1) / (d8-d0-x+d11))-90°≦φv+θv+θlv (When h1 ≧ h3) φh-θh-θlh≦Atan((w6+z) / (h1-h3))≦φh+θh+θlh (When h1<h3) φh-θh-θlh≦Atan((h3-h1) / (w6+z))+90°≦φh+θh+θlh 2. The moving body according to claim 1, wherein the above formula satisfies the above formula.

9. An imaging device including an optical system that forms an optical image on a light receiving surface of an imaging element, the optical image having a low resolution area in the vicinity of an optical axis and a high resolution area outside the low resolution area, A moving body that is installed at an installation position where the high-resolution area of ​​the optical system satisfies the following first condition, second condition, and third condition, the optical system is configured such that a projection characteristic corresponding to the low resolution region is different from a projection characteristic corresponding to the high resolution region; The first condition is: When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side is φv, the angle of view of the high resolution area is θv, the angle of view of the low resolution area is θlv, a predetermined distance behind the driver's viewpoint is d1, the distance between the viewpoint position and the installation position is x, and the height of the installation position is h1, φv-θv-θlv<-90°<φv-θlv φv-θv-θlv≦-Atan((x+d1) / h1)≦φv+θv+θlv The condition is satisfied. The second condition is: When the moving body is viewed from the front, the angle of the optical axis direction from the vertical direction is φh, the angle of view of the high resolution area is θh, and the angle of view of the low resolution area is θlh, φh-θh-θlh≦-90°≦φh-θlh φh+θlh≦90°≦φh+θh+θlh The condition is satisfied. The third condition is: When the distance from the viewpoint position to the tip of the moving body is d0, the length of the moving body is d8, the distance from the moving body to a predetermined position on the near side to be confirmed rearward is d10, the distance from the moving body to a predetermined position on the far side to be confirmed rearward is d11, the width of the moving body is w6, the height of an object to be confirmed rearward of the moving body is h3, and the distance between the installation position and the side of the moving body is z, φv-θv-θlv≦-Atan((d8-d0-x+d10) / h1)≦φv+θv+θlv (When h1 ≧ h3) φv-θv-θlv≦-Atan((d8-d0-x+d11) / (h1-h3))≦φv+θv+θlv (When h1<h3) φv-θv-θlv≦-Atan((h3-h1) / (d8-d0-x+d11))-90°≦φv+θv+θlv (When h1 ≧ h3) φh-θh-θlh≦Atan((w6+z) / (h1-h3))≦φh+θh+θlh (When h1<h3) φh-θh-θlh≦Atan((h3-h1) / (w6+z))+90°≦φh+θh+θlh 2. The moving body according to claim 1, wherein the condition satisfies the following:

10. The display device further includes a display unit that displays the side and rear of the moving object on a first display unit and displays the front of the moving object on a second display unit based on the image generated by the first imaging device.

10. The moving body according to claim 1 or 9.

Images