Movable body

Abstract

To improve imaging performed around a vehicle.SOLUTION: A movable body of an embodiment of the present invention is a movable body having an imaging apparatus comprising an optical system that forms an optical image having a high resolution region on an off-axis side on a light receiving surface of an imaging unit. The imaging apparatus is arranged to photograph the front of the movable body, lateral sides of the movable body, vertically downward of the lateral sides of the movable body, and the vertically downward of the front of the movable body. The imaging apparatus is arranged to pick up an image of the front of the movable body in the high resolution region of the optical system.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a moving body capable of imaging the surroundings by an imaging device.

Background Art

[0002] In recent years, CMS (Camera Monitor system) technology for replacing mirrors mounted on vehicles such as automobiles with electronic mirrors has been promoted. A plurality of imaging devices (cameras) installed on a vehicle for CMS may be installed so as to be able to photograph (image) in all directions as the eyes of an electronic system for safety confirmation. Also, in the United Nations regulation UN-R46 (regulation regarding indirect vision of automobiles), the driver's blind spot is defined, and CMS is also required to perform imaging and installation that lead to reduction of the blind spot, similar to existing mirrors.

[0003] Also, in a vehicle equipped with a MOIS (Moving Off Infomation System) function, a camera is installed in front of the vehicle so as to be able to detect pedestrians and bicycles in the blind spot in front of the vehicle. Also, in a vehicle equipped with a side collision detection device, a camera and a sensor are installed so as to be able to detect pedestrians and bicycles on the side or in front of the vehicle. Also, in a vehicle equipped with a CTA (Cross Traffic Alert) function, cameras capable of imaging the left and right directions in front of and behind the vehicle with high resolution are installed so as to be able to detect the approach of other vehicles crossing in front of or behind the vehicle. Patent Document 1 discloses imaging the rear of a vehicle with a camera equipped with an optical system capable of imaging a part of an imaging area with high resolution.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, cameras with high resolution in a part of the imaging area used in Patent Document 1 cannot image a wide range with high resolution. Therefore, when it is necessary to image a wide range with high resolution, it was necessary to prepare a large number of cameras so that they could cover the wide range. For this reason, conventionally, there has been room for improvement in imaging the periphery of a vehicle.

[0006] An object of the present invention is to improve imaging of the periphery of a vehicle.

Means for Solving the Problems

[0007] A moving body according to an embodiment of the present invention is a moving body having an imaging device including an optical system that forms an optical image having a high-resolution region outside the optical axis on a light-receiving surface of the imaging unit, wherein the imaging device is arranged to photograph the front of the moving body, the side of the moving body, vertically downward on the side of the moving body, and vertically downward in front of the moving body, and the imaging device is arranged to image the front of the moving body in the high-resolution region of the optical system.

Effects of the Invention

[0008] The present invention can improve imaging of the periphery of a vehicle.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each figure, the same member or element is given the same reference number, and duplicate explanations are omitted or simplified.

[0011] [Embodiment 1] In Embodiment 1, a method of installing an imaging device that captures images of an electronic side mirror video, a forward CTA video, and a forward monitoring device video at high resolution will be described. FIG. 1 is a diagram for explaining the positional relationship between the imaging device and the vehicle in Embodiment 1.

[0012] FIG. 1 is a diagram for explaining the positional relationship between the imaging device and the vehicle according to Embodiment 1 of the present invention. FIG. 1 shows an example in which an imaging device 21 is installed in front of the left side of a vehicle 1, which is an automobile, and an imaging device 22 is installed in front of the right side. Vehicle 1 is an example of a moving body. In the present embodiment, it will be described assuming that imaging devices are installed on the left and right sides of vehicle 1, but in the present invention, the imaging device may be installed on only one of them. Further, in the present embodiment, imaging devices 21 and 22 are shown as in-vehicle cameras.

[0013] The imaging devices 21 and 22 are installed so as to be able to image the side surface of the vehicle 1 in order to image the video for the electronic side mirror. Also, the imaging devices 21 and 22 are installed so as to be able to image the front of the vehicle 1 so as to be able to image the front video. Also, the imaging devices 21 and 22 are installed so as to be able to image the front lower video for the front monitoring device of the vehicle 1. The imaging devices 21 and 22 are arranged so as to be able to photograph the front of the vehicle 1, the side of the vehicle 1, the vertically downward direction on the side of the vehicle 1, and the vertically downward direction in front of the vehicle 1. Hereinafter, the imaging device 21 will be described, but since the configuration of the imaging device 22 is the same, the description of the imaging device 22 will be omitted.

[0014] FIG. 2 is a view of the imaging device according to Embodiment 1 of the present invention and the vehicle 1 as seen from the side. FIG. 2 is a schematic view of the vehicle 1 according to Embodiment 1 and the imaging device 21 arranged thereon. The vehicle 1 includes an in-vehicle system (driving support device) (not shown) for assisting a user (such as a driver or a passenger) (not shown) using the image acquired by the imaging device 21. In the present embodiment, a case where the imaging device 21 is installed on the side of the vehicle 1 is shown.

[0015] The imaging device 21 has an optical system 21a and an imaging unit 21b. The optical system 21a is an optical system in which the imaging magnification is different between a first angle of view (first field of view) 210 and a second angle of view (second field of view) 211 on the peripheral side of the first angle of view 210. The imaging surface (light receiving surface) of the imaging unit 21b includes a first region that images an object included in the first angle of view 210 and a second region that images an object included in the second angle of view 211. At this time, the number of pixels per unit angle of view in the second region is larger than the number of pixels per unit angle of view in the first region. In other words, the resolution in the second angle of view 211 (second region) of the imaging device 21 is higher than the resolution in the first angle of view 210 (first region).

[0016] Figs. 3(A) and 3(B) are diagrams for explaining the optical characteristics of the imaging device 21 according to Embodiment 1 of the present invention. With reference to Figs. 3(A) and 3(B), the optical characteristics of the optical system 21a will be described. Fig. 3(A) is a diagram showing the image height y at each half field angle on the imaging surface (light receiving surface) of the imaging unit 21b in a contour line shape. Fig. 3(B) is a diagram showing the relationship (projection characteristics of the optical system 21a) between the half field angle θ and the image height y in the first quadrant of Fig. 3(A).

[0017] As shown in Fig. 3(B), the optical system 21a is configured such that its projection characteristics y(θ) are different for an angle of view less than a predetermined half field angle θa and an angle of view greater than or equal to the half field angle θa. Therefore, when the increase amount of the image height y with respect to the half field angle θ per unit is defined as the resolution, the optical system 21a is configured such that the resolution varies depending on the angle of view (the region on the light receiving surface of the imaging unit). This local resolution can be represented by the differential value dy(θ) / dθ of the projection characteristics y(θ) at the half field angle θ. For example, it can be said that the higher the slope of the projection characteristics y(θ) in Fig. 3(B), the higher the resolution. Also, in Fig. 3(A), it is shown that the higher the resolution, the larger the interval between the contour lines of the image height y at each half field angle.

[0018] The optical system 21a of the present embodiment has projection characteristics such that the increase rate of the image height y (the slope of the projection characteristics y(θ) in Fig. 3(B)) is small in the central region near the optical axis, and the increase rate of the image height y increases as the angle of view increases in the peripheral region outside the central region.

[0019] In Fig. 3(A), a first region 10a including the center corresponds to an angle of view less than the half field angle θa, and a second region 10b outside the first region corresponds to an angle of view greater than or equal to the half field angle θa. Also, the angle of view less than the half field angle θa corresponds to the first angle of view 210 in Fig. 2, and the angle of view greater than or equal to the half field angle θa corresponds to the second angle of view 211 in Fig. 2.

[0020] As described above, the first region 10a is a region with relatively low resolution, and the second region 10b is a region with relatively high resolution. Thus, the optical system of the present embodiment realizes a high-resolution region that is highly resolved with respect to the central region and a low-resolution region near the center.

[0021] Furthermore, the characteristics shown in FIGS. 3(A) and 3(B) are merely examples, and the present invention is not limited thereto. For example, the low-resolution region and the high-resolution region of the optical system do not have to be concentric, and each region may have a distorted shape. Also, the center of gravity of the low-resolution region and the center of gravity of the high-resolution region do not have to coincide. Further, the center of gravity of the low-resolution region and the center of gravity of the high-resolution region may be deviated from the center of the light-receiving surface of the imaging unit. In the optical system of the present embodiment, it is sufficient that the low-resolution region is formed near the optical axis and the high-resolution region is formed on the peripheral side near the optical axis.

[0022] The optical system 21a is configured to satisfy Equation 1, where f is the focal length, θ is the half field angle, y is the image height on the image plane, y(θ) represents the projection characteristic representing the relationship between the image height y and the half field angle θ, and θmax is the maximum half field angle of the optical system. That is, the optical system 21a is configured such that the projection characteristic y(θ) is different from 2ftan(θ / 2) (stereographic projection method). 0.2 < 2×f×tan(θmax / 2) / y(θmax) < 0.92 ··· (Equation 1)

[0023] In an optical system having such optical characteristics, by adjusting the projection characteristic y(θ), the magnification in the radial direction with respect to the optical axis can be adjusted. As a result, the aspect ratio in the radial direction and the circumferential direction with respect to the optical axis can be controlled, and thus, unlike a conventional fish-eye lens or the like, a high-resolution image with less distortion can be obtained in the peripheral region while having a wide angle of view.

[0024] Also, by satisfying Equation 1, the resolution in the second region 10b can be made higher than that of an optical system using the stereographic projection method. If the upper limit of Equation 1 is exceeded, the resolution in the second region 10b becomes low, and the difference from the resolution in the first region 10a becomes small, which is not preferable. If the lower limit of Equation 1 is not reached, it becomes difficult to correct various aberrations such as field curvature favorably, which is not preferable.

[0025] Note that the above-mentioned Equation (1) is just an example, and the optical system in this embodiment is not limited thereto.

[0026] By configuring the optical system as described above, high resolution can be obtained in the high-resolution region. On the other hand, in the low-resolution region, the increase amount of the image height y with respect to the half field angle θ per unit is reduced, and it becomes possible to image a wider field angle. Therefore, while imaging a wide field angle equivalent to that of a fish-eye lens, a high resolution can be obtained in the high-resolution region.

[0027] Also, in this embodiment, in the high-resolution region, the projection characteristic is a stereoscopic projection method, which is an approximation of the projection characteristic of a general imaging optical system (y = 2 × f × tan(θ / 2)). Therefore, in the high-resolution region, it is possible to generate a detailed image.

[0028] Next, the configuration of the image processing system in this embodiment will be described with reference to FIG. 4. FIG. 4 is a block diagram showing the configuration of an image processing system 100 according to Embodiment 1 of the present invention. FIG. 4 is a diagram for explaining the image processing system 100 according to Embodiment 1. The image processing system 100 is included in an in-vehicle system (driving support device) (not shown). The imaging device 21 includes an optical system 21a and an imaging unit 21b. The imaging device 22 includes an optical system 22a and an imaging unit 22b.

[0029] The optical systems 21a and 22a of the imaging devices 21 and 22 are each composed of one or more optical lenses, and each has optical characteristics as shown in FIGS. 2(A) and (B). Further, the optical systems 21a and 22a form (image) their respective optical images on the light receiving surfaces of the imaging units 21b and 22b. The imaging units 21b and 22b function as imaging means, photoelectrically convert the imaged optical images, and output imaging signals. Note that, for example, color filters of R, G, and B are arranged for each pixel in a Bayer array on the light receiving surfaces of the imaging units 21b and 22b, and R, G, and B pixel signals are sequentially output as imaging signals from the imaging unit 21.

[0030] The camera processing units 31 and 32 have the function of processing the imaging signals output from the imaging units 21b and 22b, respectively. The camera processing units 31 and 32 each have an image processing unit 31a, 32a, and a camera information unit 31b, 32b.

[0031] Inside the camera processing units 31 and 32, a CPU as a computer and a memory storing a computer program as a storage medium are built-in, and the CPU executes the computer program in the memory. Thereby, it is configured to execute the processing within the camera processing units 31 and 32. The CPU is an abbreviation for Central Processing Unit.

[0032] Note that the camera processing units 31 and 32 are not limited to the above configuration, and the image processing units 31a and 32a may be configured by hardware such as a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP). ASIC is an abbreviation for Application Specific Integrated Circuit. DSP is an abbreviation for Digital Signal Processor.

[0033] The image processing units 31a and 32a perform various image correction processes on the imaging signals output from the imaging units 21b and 22b, such as white balance adjustment, gain / offset adjustment, gamma processing, color matrix processing, reversible compression processing, and distortion correction processing.

[0034] Also, for example, the image data input from the imaging units 21b and 22b according to the Bayer array is each de-Bayer processed and converted into RGB raster format image data. Note that a part of the above image processing may be performed by other blocks of the image processing system 100 other than the image processing units 31a and 32a.

[0035] The camera information units 31b and 32b function as holding means for holding camera information such as characteristic information regarding the characteristics of the optical image and the position and orientation information of the imaging devices, and respectively hold in advance in the memory the camera information of the imaging devices 21 and 22. Further, the camera information units 31b and 32b may hold information from various sensors provided in the imaging devices 21 and 22 etc.

[0036] The camera information held in the camera information units 31b and 32b includes, for example, characteristic information of the optical systems 21a and 22a, and the number of pixels of the imaging units 21b and 22b. Further, the camera information includes information such as the attachment position information and orientation (pitch, roll, yaw, etc.), the optical axis direction, and the imaging range in the vehicle coordinates of the imaging devices 21 and 22. Further, the camera information may include information such as the gamma characteristics, sensitivity characteristics, frame rate, and image format of the video output from the camera processing unit 31.

[0037] Since the attachment position information of the imaging device is often determined for each imaging device with respect to the vehicle, it is stored in advance in the memory in the camera information unit as the relative coordinates with respect to the vehicle 1. Further, the camera information may be unique information of the imaging units 21b and 22b (for example, aberration characteristics of the lens and noise characteristics of the imaging unit, etc.). The camera information is transmitted to the integration processing unit 40 and is referred to when performing image processing etc. necessary for display on the display unit in the integration processing unit 40.

[0038] The integration processing unit 40 has a function of displaying the video signals obtained from the imaging devices 21 and 22 on display devices such as the first display unit 50 and the second display unit 51, and a function of notifying the warning display unit 52 and the voice notification unit 53. Further, the integration processing unit 40 has an image recognition function etc. for an object straddling the boundary of the imaging ranges of the imaging devices 21 and 22.

[0039] The integrated processing unit 40 includes a SOC / FPGA 41, a CPU 42 as a computer, and a memory 43 as a storage medium. SOC is an abbreviation for System On Chip. FPGA is an abbreviation for Field Programable Gate Array. In this embodiment, the integrated processing unit 40 is described as being housed in a housing separate from the imaging device.

[0040] Note that some or all of the functional blocks included in the integrated processing unit 40 and the like may be realized by hardware or may be realized by the CPU 42. As the hardware, an application specific integrated circuit (ASIC), a processor (reconfigurable processor, DSP), or the like can be used. The CPU 42 performs various controls for the entire image processing system 100 by executing a computer program stored in the memory 43.

[0041] The SOC / FPGA 41 includes an image processing unit 41a, a recognition unit 41b, and an integrated control unit 41c. The image processing unit 41a has a function of performing image processing such as resolution conversion for display on a display device based on each image signal and camera information acquired from the camera processing units 31 and 32.

[0042] As described above, the camera information includes the optical characteristics of the optical systems 21a and 22a, the number of pixels of the imaging units 21b and 22b, photoelectric conversion characteristics, γ characteristics, sensitivity characteristics, format information of the image signal, coordinates of the mounting position of the imaging device in the vehicle coordinates, attitude information, and the like.

[0043] For example, the image processing unit 41a synthesizes the image signals of the respective low-resolution regions 10b of the imaging units 21b and 22b that have undergone distortion correction and the image signals of the high-resolution regions 10b so as to smoothly connect them to form an overall image for each of the imaging units 21b and 22b. Also, in the image processing unit 41a, image processing such as rotation of the image is performed based on the camera arrangement position and attitude information in the camera information, and the image signal is transmitted to the display device.

[0044] The recognition unit 41b performs image recognition processing on a composite image formed by combining the images from the imaging units 21b and 22b. The recognition unit 41b performs image recognition on a predetermined object (such as a vehicle, a person, an obstacle, etc.) in the composite image obtained from the imaging units 21b and 22b and generates an image recognition result.

[0045] In addition, the integrated control unit 41c also has a function of forming a video signal for displaying a desired image on a display device such as the first display unit 50 and the second display unit 51 from the entire processed images of each of the imaging units 21b and 22b. Further, the integrated control unit 41c generates a frame for highlighting the recognized object, CG for information such as the type, size, position, speed, etc. of the object, and warnings. CG is an abbreviation for Computer Graphics.

[0046] Furthermore, the integrated control unit 41c may generate CG of a boundary image for displaying a boundary based on the characteristic information of the optical system such as the display resolution boundary information acquired from the camera information units 31b and 32b. Then, the integrated control unit 41c performs display processing such as superimposing these CGs and characters on the image. Here, the first display unit 50, the second display unit 51, etc. function as display means and display an image signal and the integrated image recognition result.

[0047] In addition, the integrated control unit 41c communicates with a travel control unit 60 (such as an ECU) via communication means (not shown) provided inside using protocols such as CAN, FlexRay (registered trademark), and Ethernet (registered trademark). ECU is an abbreviation for Electronic Control Unit. CAN is an abbreviation for Controller Area Network. The integrated control unit 41c thereby performs display processing for appropriately changing the information to be displayed based on the vehicle control signal received from the travel control unit 60, etc. That is, the integrated control unit 41c can change, for example, the range of the image to be displayed on the display means according to the moving state of the vehicle acquired by the vehicle control signal.

[0048] The traveling control unit 60 is mounted on the vehicle 1 and is a unit incorporating a computer and memory for comprehensively performing drive control, direction control, etc. of the vehicle 1. The traveling control unit 60 outputs, as vehicle control signals, information regarding the traveling (moving state) of the vehicle 1 such as, for example, traveling speed, traveling direction, shift lever, shift gear, state of the turn signal, direction of the vehicle by means of a geomagnetic sensor, etc., and GPS information, to the integration processing unit 40.

[0049] Furthermore, the integrated control unit 41c may have a function of transmitting information such as the type, position, moving direction, moving speed, etc. of a predetermined object (such as an obstacle) recognized by the recognition unit 41b to the traveling control unit 60. Thereby, the traveling control unit 60 performs control necessary for avoiding obstacles such as stopping, driving, and changing the traveling direction of the vehicle 1. Note that the traveling control unit 60 functions as moving control means for controlling the movement of the vehicle based on the integrated image recognition result.

[0050] The first display unit 50 is, for example, a display device that is installed with a display screen facing the rear of the vehicle near the center in the vehicle width direction at the upper front of the driver's seat of the vehicle 1 and functions as an electronic rearview mirror and an electronic side mirror.

[0051] Furthermore, the first display unit 50 may be configured to include a touch panel and operation buttons to acquire an instruction from the user and output it to the integrated control unit 41c. Furthermore, the first display unit 50 can also be used, for example, as a display device that displays an electronic side mirror for checking obstacles on the left and right and a video for front CTA instead of an optical side mirror. The first display unit 50 receives and displays a video signal of an angle of view necessary for display according to the use of the first display unit 50, generated by the integrated control unit 41c.

[0052] The second display unit 51 is installed, for example, near the center in the vehicle width direction in front of the driver's seat of the vehicle 1, and functions as a display device for displaying, for example, downward in the front direction of the vehicle. Similar to the first display unit, the second display unit 51 also functions as a display device that receives a necessary imaging area generated according to the application. The second display unit 51 receives and displays a video signal with an angle of view necessary for display according to the application of the second display unit 51, which is generated by the integrated control unit 41c.

[0053] For example, the second display unit 51 can also display a navigation system, an audio system, and various control signals from the driving control unit 60. Further, the second display unit 51 may be configured to include a touch panel and operation buttons and be capable of acquiring instructions from the user.

[0054] Further, the second display unit 51 may be, for example, a display unit of a tablet terminal, and can be displayed by being connected to the integrated processing unit 40 by wire, or can also receive and display an image wirelessly via the communication unit 62. Incidentally, as the display elements of the first display unit 50 and the second display unit 51, a liquid crystal display, an organic EL display, etc. can be used, and the number of display units is not limited to one.

[0055] The integrated control unit 41c determines whether a moving object is included in the image based on the recognition result output by the recognition unit 41b, and outputs the recognition result. The moving object here is, for example, a bicycle, a pedestrian, or other vehicles, and in this embodiment, it is called a detection target. The recognition result output by the integrated control unit 41c includes the presence or absence of the detection target, the type, coordinates, and speed information of the detection target.

[0056] The warning display unit 52 functions as a warning display means, and gives, for example, a side collision warning to the driver by visual information based on the recognition result output from the integrated control unit 41c. The warning display unit 52 may be configured, for example, by an LED, and may be configured to light up or blink when information indicating the presence of a detection target is included in the recognition result.

[0057] Further, the warning display unit 52 may be configured by a display such as a liquid crystal. In this case, when the information indicating the presence of the detection target is included in the recognition result, icons, character information, etc. are output on the display. Further, the warning display unit 52 can be installed, for example, near the vehicle width direction end in front of the driver's seat of the vehicle 1 with the display screen facing the driver direction.

[0058] Further, the warning display unit 52 may be configured to be installed, for example, near the first display unit 50 or the second display unit 51, or may be configured to be substituted by the first display unit 50 or the second display unit 51.

[0059] The voice notification unit 53 functions as a voice notification means and outputs a sound based on the recognition result output from the integrated control unit 41c. For example, a speaker can be used to output a sound to notify the driver. The voice notification unit 53 is preferably installed, for example, near the vehicle width direction end in front of the driver's seat of the vehicle 1.

[0060] Further, the integrated control unit 41c also functions as a warning condition determination means, and controls to change the content of the warning output by the warning display unit 52 or the voice notification unit 53 based on the coordinates and speed of the detection target obtained by the recognition unit 41b. Further, the integrated control unit 41c may be configured to control the warning display unit 52 so that the warning level becomes higher as the distance from the detection target is closer. Further, the integrated control unit 41c may be configured to control not to give a warning notification when the detection target is moving away from the vehicle 1.

[0061] It is desirable that the integrated control unit 41c controls the voice notification unit 53 so that the notification volume increases as the distance between the detection target and the side of the vehicle 1 is closer. Incidentally, the integrated control unit 41c determines whether the traveling speed of the vehicle 1 input from the traveling control unit 60 is equal to or less than a predetermined value, and may be configured to give a warning display by the warning display unit 52 or a warning notification by the voice notification unit 53 only when it is equal to or less than the predetermined value.

[0062] When the vehicle 1, i.e., the own vehicle, is traveling at a high speed, there is a possibility that the detection target cannot be correctly recognized. Therefore, by adopting the configuration as in this embodiment, the possibility of correctly performing side detection can be improved. At this time, the predetermined value (threshold for whether to issue a warning) of the traveling speed of the vehicle 1 is desirably set to, for example, 30 km / h.

[0063] Also, when the detection target is moving compared to when it is stationary, the possibility of colliding with the vehicle 1 is high. Therefore, the integrated control unit 41c may be configured to determine whether the speed of the detection target is within a predetermined range, and to perform a warning display on the warning display unit 52 or a warning notification on the voice notification unit 53 only when it is determined that the speed of the detection target is within the predetermined range. In this case, as the predetermined range of the speed of the predetermined detection target, it is desirable to set it to, for example, 5 km / h or more and 20 km / h or less.

[0064] Also, the integrated control unit 41c determines whether the vehicle 1 is turning right or left based on the movement direction information of the vehicle 1 output from the travel control unit 60, and performs a warning display on the warning display unit 52 or a warning notification on the voice notification unit 53 only when the vehicle 1 is turning right or left.

[0065] In addition, in this embodiment, the integrated processing unit 40 is arranged in the vehicle 1. However, some of the processes of the image processing unit 41a, the recognition unit 41b, and the integrated control unit 41c of the integrated processing unit 40 may be performed by an external server or the like via a network, for example. In that case, for example, the imaging units 21b and 22b as the image acquisition means are mounted on the vehicle 1, but some of the functions of, for example, the camera processing units 31 and 32 and the integrated processing unit 40 can be processed by an external server or the like. Also, it is possible to give some or all of the functions of the integrated processing unit 40 to the travel control unit 60.

[0066] The storage unit 61 records the overall images for each of the imaging units 21b and 22b generated by the integrated processing unit 40. Further, the storage unit 61 records a predetermined frame, characters, CG such as warnings, etc. indicating the recognized object, and the images with the CG superimposed and displayed on the first display unit 50, the second display unit 51, etc., together with the time, GPS information, etc. The integrated processing unit 40 can also reproduce the past information recorded in the storage unit 61 and display it on the first display unit 50 or the second display unit 51.

[0067] The communication unit 62 is for communicating with an external server etc. via a network, and can transmit the information before being recorded in the storage unit 61, the past driving history information etc. recorded in the storage unit 61 to an external server etc. and store them in the external server etc. Also, the communication unit 62 can transmit the image to an external tablet terminal etc. as described above and display the image on the second display unit 51 which is the display unit of the tablet terminal. Also, the communication unit 62 can acquire (receive) traffic jam information and various types of information from an external server etc. and display them on the first display unit 50 or the second display unit 51 via the integrated processing unit 40.

[0068] The operation unit 63 is for inputting various instructions to the image processing system by the user's operation. The operation unit 63 includes, for example, a touch panel and operation buttons.

[0069] Next, the relationship between the characteristics and the arrangement of the imaging device 21 installed on the left front side of the vehicle 1 and the imaging device 22 installed on the right front side will be described. In the first embodiment, with the driver's seat of the vehicle 1 on the right side in the vehicle traveling direction and the passenger seat on the right side in the vehicle traveling direction, the imaging device 21 is installed on the passenger seat side.

[0070] Figs. 5(A) and 5(B) are diagrams for explaining the relationship between the characteristics of the optical system of the imaging device 21 installed in the vehicle 1 of the first embodiment and the imaging regions necessary for the display for the electronic side mirror and the front CTA. Fig. 5(A) is a side view of the vehicle 1 seen from the side, and Fig. 5(B) is a front view of the vehicle seen from the front.

[0071] In FIGS. 5(A) and 5(B), the X, Y, and Z directions are based on the installation position of the imaging device 21 as the origin. The X direction is the horizontal vehicle front direction as shown in FIG. 5(A). The Z direction is the horizontal left direction toward the vehicle 1 as shown in FIG. 5(B). The Y direction is the vertical upward direction as shown in FIGS. 5(A) and 5(B). In the X, Y, and Z directions, the direction indicated by the arrow is the positive (+) direction, and the opposite direction is the negative (-) direction.

[0072] In the XY plane of FIG. 5(A), with the vertical direction as the reference, the angle at which the -Y side of the Y-axis is directed in the +X direction around the origin is defined as the positive angle, and the angle directed in the -X direction is defined as the negative angle. In the YZ plane of FIG. 5(B), with the vertical direction as the reference, the angle at which the -Y side of the Y-axis is directed in the +Z direction around the origin is defined as the positive angle, and the angle directed in the -Z direction is defined as the negative angle. The viewpoint position E in the XY plane of FIG. 5(A) represents the driver's viewpoint position, which is the position of the eyes at the driver's seating position or the central position of the driver's seat surface.

[0073] Using the above definitions, the relationship between the optical axis and the imaging angle of view of the imaging device 21 will be described. φv, θv, and θlv in the XY plane of FIG. 5(A) will be described. φv is the direction of the optical axis of the imaging device 21 and is the angle from the vertical direction. In this embodiment, the optical axis of the imaging device is downward from the installation position. θv is the angle of view of the high-resolution region 10b of the imaging device 21. θlv is the half angle of view of the low-resolution region 10a of the imaging device 21.

[0074] φh, θh, and θlh in the YZ plane of FIG. 5(B) will be described. φh is the direction of the optical axis of the imaging device 21 and is the angle from the vertical direction. θh is the angle of view of the high-resolution region 10b of the imaging device 21. θlh is the half angle of view of the low-resolution region 10a of the imaging device 21.

[0075] Using the above definitions, the positional relationship of the imaging device 21 will be described. The distance x in the XY plane of FIG. 5(A) is the horizontal distance between the imaging device 21 and the viewpoint position E, which is the driver's viewpoint position. The distance h1 is the vertical distance between the imaging device 21 and the ground.

[0076] The distance z in the YZ plane of FIG. 5(B) is the distance between the installation position of the imaging device 21 (the first installation position) and the side surface of the vehicle 1. Here, the side under-mirror and the imaging area required for under-mirror display (the defined area of the vehicle's blind spot) will be described with reference to FIGS. 5(A), 5(B), and 6.

[0077] FIG. 6 is a plan view of the vehicle 1 of Embodiment 1 as viewed from above. The X and Z directions in FIG. 6 correspond to FIG. 5 with the installation position of the imaging device 21 as the origin. The defined area of the vehicle's blind spot is the area on the ground indicated by the shaded portions in FIGS. 5(A) and 6, and is the area that the driver needs to check with the vehicle's side under-mirror and under-mirror.

[0078] Also, the position of the corner farther from the vehicle side surface in front of the vehicle in the defined area of the vehicle's blind spot under the vehicle side surface is defined as position F1. Also, the position of the corner closer to the vehicle side surface in front of the vehicle in the defined area of the vehicle's blind spot under the vehicle side surface is defined as position F2. Also, the position of the corner at a distance w3 from the vehicle side surface in the defined area of the vehicle's blind spot under the vehicle front surface is defined as position F3. Also, the position at a distance d3 from the vehicle front corner in the defined area of the vehicle's blind spot under the vehicle front surface is defined as position F4.

[0079] By comparing position F1 and position F3, the one farther from the vehicle 1 needs to be within the viewing angle of the imaging device 21. Also, for position F2 and position F4, since position F2 is a side blind spot and position F4 is a front blind spot, position F4 is in front.

[0080] The defined area of the vehicle's blind spot is the area on the ground surface defined by widths w1 and w2 from the vehicle body side surface at distances d0 from the viewpoint position E to the vehicle tip, distances d1 and d2 from the viewpoint position E, and distance d3 from the vehicle tip.

[0081] In addition to Equation 1, the imaging device 21 is installed so that the imaging angle includes the positions at distances d1, d0 + d3, width w1, and width w2 from the viewpoint position E so that the defined area of the vehicle's blind spot is included in the imaging angle of the imaging device 21.

[0082] That is, it is installed so as to satisfy the conditions (Equation 2 and Equation 3) that the angles in the directions of the regions separated from the viewpoint position E by distances d1 and d0 + d3 are included within the imaging angle of view (φv - θv - θlv) to (φv + θv + θlv) of the imaging device 21. The angles in the directions of the regions separated from the viewpoint position E by distances d1 and d0 + d3 are (-Atan((x + d1) / h1)) to (Atan((-x + d0 + d3) / h1)). φv - θv - θlv ≤ -Atan((x + d1) / h1) ≤ φv + θv + θlv ····· (Equation 2) φv - θv - θlv ≤ Atan((-x + d0 + d3) / h1) ≤ φv + θv + θlv ····· (Equation 3)

[0083] Also, it is installed so as to satisfy the conditions (Equation 4 to Equation 6) that the angles in the directions of the regions separated from the viewpoint position E by widths w1 and w2 are included within the imaging angle of view (φh - θh - θlh) to (φh + θh + θlh) of the imaging device 21. The angles in the directions of the regions separated from the viewpoint position E by widths w1 and w2 are (-Atan((w1 - z) / h1)) to (Atan((w2 + z) / h1)). φh - θh - θlh ≤ Atan((w2 + z) / h1) ≤ φh + θh + θlh ····· (Equation 4) φh - θh - θlh ≤ -Atan((w1 - z) / h1) ≤ φh + θh + θl (when w1 ≥ w3) ····· (Equation 5) φh - θh - θlh ≤ -Atan((w3 - z) / h1) ≤ φh + θh + θl (when w1 < w3) ····· (Equation 6)

[0084] By installing the imaging device 21 so as to satisfy the above Equations 2 to 6, the specified region of the blind spot near the vehicle can be imaged. The installation conditions of the above Equations 2 to 6 are taken as the first conditions.

[0085] Next, with reference to FIGS. 7 and 8, the imaging regions (vehicle front detection region, vehicle side collision detection region) necessary for the imaging device 21 to detect a detection target in the front will be described.

[0086] FIG. 7 is a plan view of vehicle 1 as seen from above for explaining the vehicle front detection area and the vehicle side collision detection area of Embodiment 1. FIG. 8 is a front view of vehicle 1 as seen from the front for explaining the relationship between imaging device 21 of Embodiment 1 and vehicle 1. The X, Y, Z directions and angles in FIGS. 7 and 8 respectively correspond to FIGS. 5(A), 5(B), and 6, and have the same relationship.

[0087] The viewpoint position E in FIGS. 7 and 8 represents the driver's viewpoint position. The positional relationship (x, h1, z1) between imaging device 21 and vehicle 1 in FIGS. 7 and 8, and the relationship between the optical axis and the imaging angle of view (φv, θv, θlv, φh, θh, θlh) are also the same as those in FIGS. 5(A) and (B).

[0088] Since it is necessary to image the detection target in the vehicle front detection area and the vehicle side collision detection area, the height of the detection target is set as height h2. When the detection target is in the vehicle front detection area and the vehicle side collision detection area, it is desirable to give a warning using warning display unit 52. Also, when the detection target is not in the vehicle front detection area and the vehicle side collision detection area, it is desirable that warning display unit 52 does not give a warning so as not to give an unnecessary warning.

[0089] The installation conditions for imaging the detection target in the vehicle side collision detection area will be described. Imaging device 21 is installed so as to include a detection target with height h2 within the imaging angle of view for imaging the detection target, and it is desirable to image a detection target far ahead with high resolution. Therefore, in Embodiment 1, the following installation conditions (Equations 7 to 10) are satisfied so that the vehicle side collision detection area is included within the range of (φv - θv - θlv) to (φv + θv + θlv) within the imaging angle of view of imaging device 21.

[0090] That is, it is installed so as to satisfy the condition (Equation 7) that the angle (Atan((d0 + d4 - x) / (h1 - h2))) in the area direction separated by (d0 + d4) from the viewpoint position E is included in the high-resolution area 10b (φv + θlv) to (φv + θv + θlv) of imaging device 21. φv + θlv ≤ Atan((d0 + d4 - x) / (h1 - h2)) ≤ φv + θv + θlv ····· (Equation 7)

[0091] Also, it is installed so as to satisfy the condition (Equation 8) that the angle (-Atan((d5 - d0 + x) / (h1 - h2))) in the direction of the region separated from the viewpoint position E by (d5 - d0) is included in the imaging angle (φv - θv - θlv) to (φv + θv + θlv) of the imaging device 21. φv - θv - θlv ≤ -Atan((d5 - d0 + x) / (h1 - h2)) ≤ φv + θv + θlv ····· (Equation 8)

[0092] Also, it is installed so as to satisfy the installation conditions (Equations 9 and 10) that the end portion of w4 and the side surface of the vehicle are included within the imaging angle of the imaging device 21. φh - θh - θlh ≤ -Atan((w4 - z) / (h1 - h2)) ≤ φh + θh + θlh ····· (Equation 9) φh - θh - θlh ≤ Atan(z / (h1 - h2)) ≤ φh + θh + θlh ····· (Equation 10)

[0093] By installing the imaging device 21 so as to satisfy the above Equations 7 to 10, it is possible to image the specified region of the vehicle side collision detection region. The installation conditions of the above Equations 7 to 10 are set as the second condition.

[0094] The installation conditions for imaging a detection target in the front with high resolution will be described. For front detection, it is desirable to image a detection target approaching from a distance in the front with high resolution.

[0095] That is, it is installed so as to satisfy the condition (Equation 11) that the angle (Atan((d0 + d6 - x) / (h1 - h2))) in the direction of the region separated from the viewpoint position E by (d0 + d6) is included in the high - resolution region 10b (φv + θlv) to (φv + θv + θlv) of the imaging device 21. φv + θlv ≤ Atan((d0 + d6 - x) / (h1 - h2)) ≤ φv + θv + θlv ····· (Equation 11)

[0096] Also, it is installed so as to satisfy the condition (Equation 12) that the angle (-Atan((x - d0) / (h1 - h2))) in the direction of the region separated from the viewpoint position E by d0 is included within the imaging angle of view (φv - θv - θlv) to (φv + θv + θlv) of the imaging device 21. φv - θv - θlv ≤ -Atan((x - d0) / (h1 - h2)) ≤ φv + θv + θlv ····· (Equation 12)

[0097] Also, it is installed so as to satisfy the installation conditions (Equations 13 and 14) that the end of w5 is included within the imaging angle of view of the imaging device 21. φh - θh - θlh ≤ -Atan(((w5 - w2) / 2 - z) / (h1 - h2)) ≤ φh + θh + θlh ····· (Equation 13) φh - θh - θlh ≤ Atan(((w5 + w2) / 2 + z) / (h1 - h2)) ≤ φh + θh + θlh ····· (Equation 14)

[0098] By installing the imaging device 21 so as to satisfy the above Equations 11 to 14, the vehicle front detection region can be imaged. The installation conditions of the above Equations 11 to 14 are set as the third condition.

[0099] Next, the installation conditions under which the imaging device 21 can image the horizontal direction with high resolution for the display for the front CTA will be described. For the display for the front CTA, it is desirable to be able to display an object approaching from a distance in the left - right direction at an intersection with poor visibility. Therefore, in Embodiment 1, the imaging device 21 is installed so that the left - right horizontal direction can be appropriately confirmed in front of the driver's viewpoint position E.

[0100] That is, since the far - away in the left - right horizontal direction extends to infinity, the imaging device is installed so that it can image an object in the vicinity to the horizontal direction with high resolution. Here, an example of the installation conditions (Equations 15 and 16) as the fourth condition under which the imaging device 21 can image at least one direction in the left - right horizontal direction in front of the driver's viewpoint position E (driver's seat) will be described.

[0101] In Embodiment 1, when the imaging device 21 is in front of the viewpoint position E (x ≥ 0), it is installed so as to satisfy Equation 15 or Equation 16 as the following installation condition (the fourth condition). That is, at least one of the high-resolution regions 10b in the left-right direction ((φh - θh - θlh) to (φh - θlh) in the right direction toward the vehicle front and (φh + θlh) to (φh + θh + θlh) in the left direction toward the vehicle front) includes the left-right horizontal direction. φh - θh - θlh ≤ -90° ≤ φh - θlh ··· (Equation 15) φh + θlh ≤ 90° ≤ φh + θh + θlh ··· (Equation 16)

[0102] By installing the imaging device 21 so as to satisfy Equation 15 and Equation 16 above, at least one of the left-right horizontal directions can be imaged with high resolution for CTA display. The installation conditions of Equation 15 and Equation 16 above are set as the fourth condition.

[0103] Hereinafter, specific values will be actually given to explain the imaging device 21 installed in front of the passenger seat side. Here, a case where the vehicle 1 is an automobile with sizes of d0 = 0.8 m and w2 = 2.4 m will be described.

[0104] In this embodiment, the defined regions of the blind spots near the vehicle are d1 = 1.75 m, d2 = 1 m, d3 = 2 m, w1 = 2 m, and w3 = 2 m. Also, the vehicle side collision detection regions are d4 = 15 m, d5 = 38.3 m, and w4 = 4.25 m. Also, the vehicle front collision region is d6 = 10 m and w5 = 3.5 m. A specific example of including this ground region in the imaging device 21 of this embodiment will be described.

[0105] For example, assume a case where a camera unit with a vertical viewing angle θv and a horizontal viewing angle θh of the high-resolution region 10b of 23.6 degrees and 66.4 degrees, respectively, is installed at positions of x = 1.1 m, z = 0.3 m, and h1 = 2.3 m. The camera unit is an example of the imaging device 21. In this case, the vertical angle φv and the horizontal angle φh of the optical axis of the imaging device 21 may be installed at -54.7° ≤ φv ≤ 40.1° and -41.6° ≤ φh ≤ 54.7°, respectively, obtained from the above Equations 2 to 6.

[0106] Also, while including the defined area of the blind spot near the vehicle and the vehicle side collision detection area within the angular range, it is desirable to include the far front in the high-resolution area 10b. For this purpose, the vertical angle φv and the horizontal angle φh of the optical axis of the imaging device 11 may be set to -3.1° ≤ φv ≤ 1.2° and -41.6° ≤ φh ≤ 11.4°, respectively, which are obtained from the aforementioned Equations 2 to 10.

[0107] Also, while including the defined area of the blind spot near the vehicle and the vehicle front detection area within the angular range, it is desirable to include the far front in the high-resolution area 10b. For this reason, the imaging device 21 is preferably installed at -4.7° ≤ φv ≤ 18.9° and -13.8° ≤ φh ≤ 54.7°, which are obtained from Equations 2 to 6 and Equations 11 to 14.

[0108] Also, while including the defined area of the blind spot near the vehicle within the angular range, in order to image the left and right horizontal directions for CTA, the imaging device 21 is preferably installed at -54.7° ≤ φv ≤ 40.1° and -23.6° ≤ φh ≤ 23.6°, which are obtained from Equations 2 to 6 and Equations 15 to 16.

[0109] As described above, by installing the imaging device 21 under the first to fourth conditions, the defined area of the blind spot near the vehicle can be imaged, and furthermore, the detection area around the vehicle can be captured.

[0110] Note that this embodiment does not necessarily need to satisfy all of the above first, second, third, and fourth conditions, and includes those installed so as to satisfy at least one of the above first and second conditions and the third and fourth conditions.

[0111] [Embodiment 2] Embodiment 2 of the present invention will describe the installation conditions when installing a camera unit so as to also serve as an electronic side mirror in addition to the configuration of Embodiment 1. Here, the imaging area (defined area behind the side of the vehicle) required for the electronic side mirror display will be described with reference to FIG. 9.

[0112] FIG. 9 is a plan view of the vehicle 1 according to Embodiment 2, with the X and Z directions having the installation position of the imaging device 21 as the origin. Also, the positional relationship (x, h1, z1) between the imaging device 21 and the vehicle 1 in FIG. 9 and the relationship between the optical axis and the imaging angle of view (φv, θv, θlv, φh, θh, θlh) are the same as those in FIGS. 5(A) and 5(B). The specified area behind the side of the vehicle is the area on the ground indicated by the shaded part in FIG. 9, which is the area that the driver needs to check with the side mirror of the vehicle.

[0113] The specified area behind the side of the vehicle is an area on the ground surface defined by a width w7 from the side surface of the vehicle body at a distance d7 behind from the viewpoint position E of the driver and a width w8 from the side surface of the vehicle body at a distance d8. Let the area on the passenger seat side in the specified area behind the side of the vehicle be region1, and the area on the driver's seat side be region2. Region1 and region2 have a shape that is line-symmetric with respect to the center line in the longitudinal direction of the vehicle 1.

[0114] The installation conditions for the imaging device 21 installed on the passenger seat side to image the specified area region1 behind the side of the vehicle while being able to image the horizontal direction with high resolution will be described. Since the far side of region1, which is the specified area behind the side of the vehicle, extends to infinity behind the vehicle, the imaging device 21 is installed so as to be able to image the horizontal direction at infinity, that is, in the negative direction of the X axis. Also, in order to check with the camera image an object far away from the driver, it is desirable to install it so as to be able to image the vicinity of the ground direction to the horizontal direction with high resolution.

[0115] Therefore, in Embodiment 2, it is installed so as to satisfy the installation condition (Equation 17) in which the horizontal angle (-90°) is included within the range of the high-resolution area 10b (φv - θv - θlv) and (the angle of φv - θlv) of the imaging device 21. φv - θv - θlv ≤ -90° ≤ φv - θlv ····· (Equation 17)

[0116] Also, in addition to Equation 17, it is installed so that the line segment PQ (area with width w7) of the specified area region1 at a position d7 behind from the viewpoint position E is included in the imaging angle of view so that the specified area region1 on the side of the vehicle is included in the imaging angle of view of the imaging device 21.

[0117] That is, it is installed so that the angle (-Atan((x + d7) / h1)) in the tip region direction of region1, which is separated by d7 from the viewpoint position E to the rear, satisfies the condition (Equation 18) included in the imaging angle (φv - θv - θlv) to (φv + θv + θlv) of the imaging device 21. φv - θv - θlv ≤ -Atan((x + d7) / h1) ≤ φv + θv + θlv ····· (Equation 18)

[0118] Also, it is installed so as to satisfy the installation condition (Equation 19) that the end portion of w8 is included within the imaging angle of the imaging device 21. φh - θh - θlh ≤ -Atan((w8 - z) / h1) ≤ φh + θh + θlh ····· (Equation 19)

[0119] By installing so as to satisfy the installation conditions (the fifth condition) of the above Equations 17 to 19, the imaging device 21 can image the specified region region1 behind the side surface on one side of the vehicle 1 while imaging the horizontal direction with high resolution. Here, the side surface on one side of the vehicle 1 is the side surface on the passenger seat side.

[0120] As described above, by satisfying the conditions of Equations 17 to 19, the rear side surface of the vehicle 1 can be included within the imaging angle. Further, in order to include the specified region of the dead angle near the vehicle, the vehicle front detection region, and the vehicle side collision detection region, in addition to the above Equations 17 to 19, the first condition, the second condition, and the third condition, it is desirable to direct the optical axis toward the ground direction. Further, in order to photograph the horizontal direction with high resolution for CTA display, it is desirable to add the fourth condition to the above Equations 17 to 19, the first condition, the second condition, and the third condition.

[0121] In addition, for large vehicles, it is desirable to be able to confirm the view of a wider range (the specified region of the wide area behind the vehicle side surface) than the specified region behind the vehicle side surface.

[0122] The defined area in the wide rear region on the side of the vehicle shall be the area on the ground surface (region3, region4). Region3 and region4 are defined by a width w7' from the side surface of the vehicle body at a distance d7' behind the viewpoint position E of the driver. Region3 and region4 are defined by a width w8' from the side surface of the vehicle body at a distance d9 behind the viewpoint position E. Region3 and region4 are defined by a width w8' from the side surface of the vehicle body at a distance d8' behind the viewpoint position E. The imaging device 21 arranged on the passenger seat side can meet the installation conditions of large vehicles by installing it so that the defined area region3 in the wide rear region on the side of the vehicle is within the imaging range.

[0123] In addition, this embodiment does not necessarily need to satisfy all of the above first condition, second condition, third condition, fourth condition, and fifth condition, and includes those installed to satisfy at least one of the above first condition and second condition, third condition, fourth condition and the fifth condition.

[0124] The installation conditions of the imaging device 21 on the passenger seat side have been described above. In the case of the imaging device 22 arranged on the driver's seat side, it is also arranged under the same installation conditions. Thereby, imaging can be performed with a smaller number of cameras.

[0125] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0126] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist.

[0127] The disclosure of this embodiment includes the following configurations. (Configuration 1) A moving body having an imaging device including an optical system that forms an optical image having a high-resolution region outside the optical axis on a light-receiving surface of the imaging unit, wherein the imaging device is arranged to photograph the front of the moving body, the side of the moving body, vertically downward on the side of the moving body, and vertically downward in front of the moving body, and in the high-resolution region of the optical system, the imaging device is arranged to image the front of the moving body. The moving body is characterized by this. (Configuration 2) The optical system has a low-resolution region near the optical axis, and has a projection characteristic in which the image height per unit angle of view in the angle of view of the high-resolution region is higher than that of the low-resolution region near the optical axis. The moving body according to Configuration 1, characterized by this. (Configuration 3) When the image height of the optical system is y and the half angle of view is θ, the projection characteristic y(θ) satisfies the following condition 0.2 < 2×f×tan(θmax / 2) / y(θmax) < 0.92 The moving body according to Configuration 1 or Configuration 2, characterized by satisfying this. (Configuration 4) The optical axis direction of the optical system is in the ground direction from the installation position of the imaging device. The moving body according to any one of Configurations 1 to 3, characterized by this. (Configuration 5) When the moving body is viewed from the side, the angle of the optical axis direction from the vertical direction is φv, the angle of view of the high-resolution region is θv, and the angle of view of the low-resolution region is θlv. When the moving body is viewed from the front, the angle of the optical axis direction from the vertical direction is φh, the angle of view of the high-resolution region is θh, and the angle of view of the low-resolution region is θlh. A predetermined distance d1 behind from the viewpoint position of the driver driving the moving body is defined. The distance between the viewpoint position and the installation position of the imaging device is x. The vertical distance between the installation position and the ground is h1. The distance between the installation position and the side surface of the moving body is z. Let the distance from the viewpoint position to the tip of the moving body be d0, Let the distance from the tip of the moving body to a predetermined position in the front lower part to be confirmed be d3, Let the distance to be confirmed from the side surface of the moving body be w1, When the width of the moving body is w2, φv - θv - θlv ≤ -Atan((x + d1) / h1) ≤ φv + θv + θlv φv - θv - θlv ≤ Atan((-x + d0 + d3) / h1) ≤ φv + θv + θlv φh - θh - θlh ≤ Atan((w2 + z) / h1) ≤ φh + θh + θlh φh - θh - θlh ≤ -Atan((w1 - z) / h1) ≤ φh + θh + θl (when w1 ≥ w3) φh - θh - θlh ≤ -Atan((w3 - z) / h1) ≤ φh + θh + θl (when w1 < w3) satisfying the moving body according to Configuration 1 or Configuration 2, characterized in that. (Configuration 6) Let the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side be φv, the angular field of view of the high-resolution region be θv, and the angular field of view of the low-resolution region be θlv. Let the predetermined distance in the front from the viewpoint position of the driver driving the moving body be d4, Let the predetermined distance in the rear from the viewpoint position be d5, Let the distance between the viewpoint position and the installation position of the imaging device be x, Let the vertical distance between the installation position and the ground be h1, Let the distance from the viewpoint position to the tip of the moving body be d0, When the height of the object to be confirmed by the moving body is h2, φv + θlv ≤ Atan((d0 + d4 - x) / (h1 - h2)) ≤ φv + θv + θlv φv - θv - θlv ≤ -Atan((d5 - d0 + x) / (h1 - h2)) ≤ φv + θv + θlv satisfying the moving body according to Configuration 1, characterized in that. (Configuration 7) Let the distance to be checked on the side of the moving body be w4, when the distance between the installation position and the side surface of the moving body is z, φh - θh - θlh ≤ -Atan((w4 - z) / (h1 - h2)) ≤ φh + θh + θlh φh - θh - θlh ≤ Atan(z / (h1 - h2)) ≤ φh + θh + θlh satisfying the moving body according to Configuration 6, characterized in that. (Configuration 8) Let the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side be φv, the angular field of view of the high-resolution region be θv, and the angular field of view of the low-resolution region be θlv. Let the predetermined distance in front from the viewpoint position of the driver driving the moving body be d6. Let the distance between the viewpoint position and the installation position of the imaging device be x. Let the vertical distance between the installation position and the ground be h1. Let the distance from the viewpoint position to the tip of the moving body be d0. when the height of the object to be checked on the moving body is h2, φv + θlv ≤ Atan((d0 + d6 - x) / (h1 - h2)) ≤ φv + θv + θlv φv - θv - θlv ≤ -Atan((x - d0) / (h1 - h2)) ≤ φv + θv + θlv satisfying the moving body according to Configuration 1 or Configuration 2, characterized in that. (Configuration 9) Let the vehicle width of the moving body be w2. Let the width to be checked in front of the moving body be w5. when the distance between the installation position and the side surface of the moving body is z, φh - θh - θlh ≤ -Atan(((w5 - w2) / 2 - z) / (h1 - h2)) ≤ φh + θh + θlh φh - θh - θlh ≤ Atan(((w5 + w2) / 2 + z) / (h1 - h2)) ≤ φh + θh + θlh satisfying the moving body according to Configuration 8, characterized in that. (Configuration 10) The side surface on one side of the moving body is the side surface on the passenger seat side of the moving body, and the moving body according to any one of Configurations 1 to 9 is characterized thereby. (Configuration 11) When the angle in the optical axis direction from the vertical direction when the moving body is viewed from the front is φh, the angle of view of the high-resolution region is θh, and the angle of view of the low-resolution region is θlh, φh - θh - θlh ≤ -90° ≤ φh - θlh φh + θlh ≤ 90° ≤ φh + θh + θlh is satisfied The moving body according to Configuration 1 or Configuration 2 is characterized thereby. (Configuration 12) When the angle in the optical axis direction from the vertical direction when the moving body is viewed from the side is φv, the angle of view of the high-resolution region is θv, and the angle of view of the low-resolution region is θlv, When the predetermined distance behind from the viewpoint position of the driver driving the moving body is d7, When the distance between the viewpoint position and the installation position of the imaging device is x, When the vertical distance between the installation position and the ground is h1, φv - θv - θlv ≤ -90° ≤ φv - θlv φv - θv - θlv ≤ -Atan((x + d7) / h1) ≤ φv + θv + θlv The moving body according to Configuration 1 or Configuration 2 is characterized by satisfying the above.

Explanation of Signs

[0128] 1 Vehicle 21 Imaging device 22 Imaging device 100 Image processing system 21a Optical system 22a Optical system 21b Imaging unit 22b Imaging unit 31 Camera processing unit 32 Camera processing unit 40 Integrated processing unit 41 SOC / FPGA 42 CPU 43 Memory 50 First display unit 51 Second display unit 52 Warning display unit 53 Voice notification unit 60 Travel control unit 61 Memory unit 62 Communication unit 63 Operation unit

Claims

1. A moving body having an imaging device equipped with an optical system that forms an optical image having a high-resolution region outside the optical axis on the light-receiving surface of the imaging unit, wherein the imaging device is arranged to photograph the front of the moving body, the side of the moving body, vertically downward on the side of the moving body, and vertically downward in front of the moving body, and the imaging device is arranged to image the front of the moving body in the high-resolution region of the optical system. A moving body characterized by the above.

2. The optical system has a low-resolution region near the optical axis and has a projection characteristic in which the image height per unit angle of view in the angle of view of the high-resolution region is higher than that in the angle of view of the low-resolution region near the optical axis. The moving body according to claim 1, characterized by the above.

3. When the image height of the optical system is y and the half angle of view is θ, the projection characteristic y(θ) satisfies the following condition 0.2 < 2 × f × tan(θmax / 2) / y(θmax) < 0.92 The moving body according to claim 1, characterized by satisfying the above.

4. The optical axis direction of the optical system is in the ground direction from the installation position of the imaging device. The moving body according to claim 1, characterized by the above.

5. When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side is φv, the angle of view of the high-resolution region is θv, and the angle of view of the low-resolution region is θlv, when the angle of the optical axis direction from the vertical direction when the moving body is viewed from the front is φh, the angle of view of the high-resolution region is θh, and the angle of view of the low-resolution region is θlh, when the predetermined distance behind from the viewpoint position of the driver driving the moving body is d1, the distance between the viewpoint position and the installation position of the imaging device is x, the vertical distance between the installation position and the ground is h1, the distance between the installation position and the side of the moving body is z, the distance from the viewpoint position to the tip of the moving body is d0, the predetermined distance in front below to be confirmed from the tip of the moving body is d3, the distance to be confirmed from the side of the moving body is w1, and the width of the moving body is w2, φv - θv - θlv ≤ -Atan((x + d1) / h1) ≤ φv + θv + θlv φv - θv - θlv ≤ Atan((-x + d0 + d3) / h1) ≤ φv + θv + θlv φh - θh - θlh ≤ Atan((w2 + z) / h1) ≤ φh + θh + θlh φh - θh - θlh ≤ -Atan((w1 - z) / h1) ≤ φh + θh + θl (when w1 ≥ w3) φh - θh - θlh ≤ -Atan((w3 - z) / h1) ≤ φh + θh + θl (when w1 < w3) satisfying The moving body according to claim 1, characterized in that.

6. When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side is φv, the angular field of view of the high-resolution region is θv, and the angular field of view of the low-resolution region is θlv, Let the predetermined distance in front from the viewpoint position of the driver driving the moving body be d4, Let the predetermined distance behind from the viewpoint position be d5, Let the distance between the viewpoint position and the installation position of the imaging device be x, Let the vertical distance between the installation position and the ground be h1, Let the distance from the viewpoint position to the tip of the moving body be d0, When the height of the object to be confirmed by the moving body is h2, φv + θlv ≤ Atan((d0 + d4 - x) / (h1 - h2)) ≤ φv + θv + θlv φv - θv - θlv ≤ -Atan((d5 - d0 + x) / (h1 - h2)) ≤ φv + θv + θlv satisfying The moving body according to claim 1, characterized in that.

7. Let the distance to be confirmed on the side of the moving body be w4, When the distance between the installation position and the side surface of the moving body is z, φh - θh - θlh ≤ -Atan((w4 - z) / (h1 - h2)) ≤ φh + θh + θlh φh - θh - θlh ≤ Atan(z / (h1 - h2)) ≤ φh + θh + θlh satisfying The moving body according to claim 6, characterized in that.

8. When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side is φv, the angular field of view of the high-resolution region is θv, and the angular field of view of the low-resolution region is θlv, Let the predetermined distance in front from the viewpoint position of the driver driving the moving body be d6, Let the distance between the viewpoint position and the installation position of the imaging device be x, Let the vertical distance between the installation position and the ground be h1, Let the distance from the viewpoint position to the tip of the moving body be d0, When the height of the object to be confirmed by the moving body is h2, φv + θlv ≤ Atan((d0 + d6 - x) / (h1 - h2)) ≤ φv + θv + θlv φv - θv - θlv ≤ -Atan((x - d0) / (h1 - h2)) ≤ φv + θv + θlv satisfying The moving body according to claim 1, characterized in that.

9. Let the vehicle width of the moving body be w2, Let the width to be confirmed in front of the moving body be w5, When the distance between the installation position and the side surface of the moving body is z, φh - θh - θlh ≤ -Atan(((w5 - w2) / 2 - z) / (h1 - h2)) ≤ φh + θh + θlh φh - θh - θlh ≤ Atan(((w5 + w2) / 2 + z) / (h1 - h2)) ≤ φh + θh + θlh Satisfying The moving body according to claim 8, characterized in that it satisfies the above conditions.

10. The moving body according to claim 1, characterized in that one side surface of the moving body is the side surface on the passenger seat side of the moving body.

11. When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the front is φh, the angle of view of the high-resolution region is θh, and the angle of view of the low-resolution region is θlh, φh - θh - θlh ≤ -90° ≤ φh - θlh φh + θlh ≤ 90° ≤ φh + θh + θlh Satisfying The moving body according to claim 1, characterized in that it satisfies the above conditions.

12. When the angle of the optical axis direction from the vertical direction when the moving body is viewed from the side is φv, the angle of view of the high-resolution region is θv, and the angle of view of the low-resolution region is θlv, When a predetermined distance behind from the viewpoint position of the driver driving the moving body is d7, When the distance between the viewpoint position and the installation position of the imaging device is x, When the vertical distance between the installation position and the ground is h1, φv - θv - θlv ≤ -90° ≤ φv - θlv φv - θv - θlv ≤ -Atan((x + d7) / h1) ≤ φv + θv + θlv The moving body according to claim 1, characterized in that it satisfies the above conditions.

Images