Imaging device, control method for imaging device, and program

Abstract

To provide an imaging device with a simple configuration that can properly warn of a possible collision even when a vehicle is loaded with a payload.SOLUTION: An imaging device (100) disposed in a vehicle (10) includes: an image capturing unit (20) disposed above the vehicle so that an image of a structure (14) in front of the vehicle in the direction of travel is formed in a first angle-of-view area (31a) and an image of a load (12) of the vehicle is formed in a second angle-of-view area (31b); a ranging unit (504) that uses the first angle-of-view area to distance the height of the structure and uses the second angle-of-view area to distance the height of the load on the vehicle; a comparing unit (505) that compares the height of the structure and the height of the load by using the distance measurement result by the ranging unit; and a warning unit (506) that processes a warning to a driver of the vehicle based on the comparison result by the comparing unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program.

Background Art

[0002] Vehicles such as trucks have a loading platform on which loads can be placed. For example, a work vehicle such as an excavator is placed on a truck and transported to a work site. At this time, the driver of the vehicle needs to drive while paying attention so that the load on the loading platform does not collide with surrounding objects or structures on the road.

[0003] On the other hand, in recent years, technologies are known in which sensors and cameras are attached to vehicles, and they are used to monitor whether the vehicle collides with surrounding objects and issue a warning if there is a possibility of collision. For example, Patent Document 1 discloses a configuration in which a camera and a measuring instrument attached to a vehicle are used to measure the height of an elevated structure provided above the road on which the vehicle travels, and a warning is issued based on the determination result of whether the vehicle can pass under the elevated structure.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the configuration disclosed in Patent Document 1, while it is possible to measure the height of the forward elevated structure, the height of the host vehicle cannot be measured. For this reason, when a load is loaded on a vehicle with a loading platform such as a truck and the load exceeds the height of the vehicle, it is difficult to determine whether the vehicle can safely pass under the elevated structure. On the other hand, if a camera and a measuring instrument are additionally mounted to measure the height of the load, the configuration becomes complicated and the cost increases.

[0006] Therefore, an object of the present invention is to provide an imaging device that can appropriately warn of the possibility of collision even when there is a load on the vehicle with a simple configuration.

Means for Solving the Problems

[0007] An imaging device according to one aspect of the present invention is an imaging device disposed on a vehicle, the imaging device including an imaging unit disposed above the vehicle to form an image of a structure in front of the vehicle in a first angular field region and an image of a load on the vehicle in a second angular field region, a distance measuring unit that measures the height of the structure using the first angular field region and measures the height of the load on the vehicle using the second angular field region, a comparison unit that compares the height of the structure and the height of the load using the distance measurement result by the distance measuring unit, and a warning unit that performs processing to warn a driver of the vehicle based on a comparison result by the comparison unit.

[0008] Other objects and features of the present invention will be described in the following embodiments.

Advantages of the Invention

[0009] According to the present invention, it is possible to provide an imaging device that can appropriately warn of the possibility of collision even when there is a load on the vehicle with a simple configuration.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] <First Embodiment> First, referring to FIG. 1, the vehicle in the first embodiment of the present invention will be described. FIG. 1 is an external view of a truck (vehicle) 10 which is an example of a cargo vehicle in this embodiment. In this embodiment, a small truck is described as an example of the vehicle, but it is not limited thereto, and the vehicle may be a large truck such as a dump truck or a trailer.

[0013] As shown in FIG. 1, the truck 10 has a loading platform 11 at its rear. The side plates of the loading platform 11 at the rear end of the vehicle are removable. When loading goods onto the loading platform 11, the side plates of the loading platform 11 can be removed and the goods can be loaded from the rear side of the truck 10. A loaded object 12 is loaded on the loading platform 11. The height of the loaded object 12 varies according to the goods to be loaded. As shown in FIG. 1, a loaded object 12 higher than the height (maximum height) of the truck 10 may be loaded.

[0014] The truck 10 is provided with an imaging unit 20. The imaging unit 20 is a camera that captures images above the front in the traveling direction of the truck 10 and above the load 12 loaded on the loading platform 11. For example, it is arranged and attached in front of the upper part of the driver's seat of the truck 10.

[0015] The control processing unit 21 includes a CPU (Central Processing Unit) that performs operations and controls, a ROM (Read Only Memory) that is a main storage device, and a RAM (Random Access Memory) (none of which are shown). The ROM stores basic setting data and a camera processing program according to this embodiment. The CPU calls a program corresponding to the processing content from the ROM and expands it in the RAM, and executes the operations of each block described later inside the control processing unit 21. The control processing unit 21 is built into, for example, the front of the truck 10.

[0016] In addition, the control processing unit 21 receives the image (captured image) captured by the imaging unit 20, executes various processes described later on the image, and transmits the result to the display unit 22. Also, the control processing unit 21 controls each unit, such as issuing a warning to the driver, according to the processing result inside the control processing unit 21. For this reason, the control processing unit 21 is electrically connected to each of the imaging unit 20 and the display unit 22 via a cable (not shown).

[0017] The display unit 22 is a monitor such as a liquid crystal display, and is a confirmation means for the driver of the truck 10 to confirm the display content. The display unit 22 mainly receives and displays the image captured by the imaging unit 20 and processed by the control processing unit 21. The display unit 22 may also use a display unit such as a car navigation system.

[0018] Next, the imaging unit 20 will be described in detail. The imaging unit 20 includes an optical system 202 and an image sensor 210. The optical system 202 is an optical system with different imaging magnifications for a first angle of view (first field of view) 30 and a second angle of view (second field of view) 31 on the peripheral side of the first angle of view 30. The imaging surface (light receiving surface) of the image sensor 210 includes a first region for imaging an object included in the first angle of view 30 and a second region for imaging an object included in the second angle of view 31. At this time, the number of pixels per unit angle of view in the second region is larger than the number of pixels per unit angle of view in the first region. In other words, the resolution in the second angle of view (second region) of the imaging unit 20 is higher than the resolution in the first angle of view (first region).

[0019] Next, with reference to FIGS. 2(A) and (B), the optical characteristics of the optical system 202 will be described. FIG. 2(A) is a diagram showing the image height y at each half angle of view on the imaging surface (light receiving surface) of the image sensor 210 in a contour line shape. FIG. 2(B) is a diagram showing the relationship (projection characteristics of the optical system 202) between the half angle of view θ (deg) and the image height y (mm) in the first quadrant of FIG. 2(A). In FIG. 2(B), the horizontal axis represents the half angle of view θ, and the vertical axis represents the image height y, respectively.

[0020] As shown in FIG. 2(B), the optical system 202 is configured such that its projection characteristics y(θ) are different for an angle of view less than a predetermined half angle of view θa and an angle of view greater than or equal to the half angle of view θa. Therefore, when the increase amount of the image height y with respect to the half angle of view θ per unit is defined as the resolution, the optical system 202 is configured such that the resolution varies depending on the angle of view (region on the light receiving surface of the image sensor). This local resolution can be represented by the differential value dy(θ) / dθ of the projection characteristics y(θ) at the half angle of view θ. For example, it can be said that the higher the slope of the projection characteristics y(θ) in FIG. 2(B), the higher the resolution. Also, in FIG. 2(A), it shows that the higher the resolution, the larger the interval between the contour lines of the image height y at each half angle of view.

[0021] The optical system 202 of the present embodiment has projection characteristics such that the increase rate of the image height y (the slope of the projection characteristics y(θ) in FIG. 2(B)) is small in the central region near the optical axis, and the increase rate of the image height y increases as the angle of view increases in the peripheral region outside the central region.

[0022] In FIG. 2(A), a first region 201a including the center corresponds to an angular extent less than a half angular extent θa, and a second region 201b outside the first region corresponds to an angular extent of θa or more. Further, the angular extent less than the half angular extent θa corresponds to a first angular extent 30 in FIG. 1, and the angular extent of θa or more corresponds to a second angular extent 31 in FIG. 1.

[0023] As described above, the first region 201a is a relatively low-resolution region, and the second region 201b is a relatively high-resolution region. Further, the first region 201a is a high-distortion region with relatively much distortion, and the second region 201b is a low-distortion region with relatively little distortion. Therefore, in the present embodiment, the first region 201a may be referred to as a low-resolution region or a high-distortion region, and the second region 201b may be referred to as a high-resolution region or a low-distortion region, respectively.

[0024] Note that the optical characteristics shown in FIG. 2 are merely examples and are not limited thereto. For example, the low-resolution region and the high-resolution region of the optical system 202 do not have to be configured concentrically, and each region may have a distorted shape. Further, the center of gravity of the low-resolution region and the center of gravity of the high-resolution region do not have to coincide. Further, the center of gravity of the low-resolution region and the center of gravity of the high-resolution region may be shifted from the center of the light-receiving surface of the imaging element 210. In the optical system of the present embodiment, it is sufficient that the low-resolution region is formed near (central side) the optical axis and the high-resolution region is formed on the peripheral side near the optical axis.

[0025] When the focal length of the optical system 202 is f (mm), the half angular extent is θ (deg), the image height on the image plane is y (mm), the projection characteristic representing the relationship between the image height y and the half angular extent θ is y(θ), and θmax (deg) is the maximum half angular extent of the optical system, it is preferable that the following conditional expression (1) is satisfied. That is, the optical system 202 is configured such that the projection characteristic y(θ) is different from 2ftan(θ / 2) (stereoscopic projection method).

[0026] 0.20 < 2×f×tan(θmax / 2) / y(θmax) < 0.92 ···(1) In an optical system having such optical characteristics, by adjusting the projection characteristic y(θ), the magnification in the radial direction with respect to the optical axis can be adjusted. As a result, the aspect ratio in the radial direction and the circumferential direction with respect to the optical axis can be controlled. Therefore, unlike a conventional fisheye lens or the like, it is possible to obtain a high-resolution image with a wide angle of view and little distortion in the peripheral region. Note that a fisheye lens is an ultra-wide-angle lens with a short focal length that can photograph a wide range of about 180 degrees.

[0027] Also, by satisfying the conditional expression (1), the resolution in the second region 201b can be made higher than that of an optical system using a stereoscopic projection method. If the upper limit value of the conditional expression (1) is exceeded, the resolution in the second region 201b becomes low, and the difference from the resolution in the first region 201a becomes small, which is not preferable. On the other hand, if the lower limit value of the conditional expression (1) is fallen below, it becomes difficult to correct various aberrations such as field curvature well, which is not preferable.

[0028] Note that the conditional expression (1) is an example, and the optical system 202 in the present embodiment is not limited thereto. The optical system 202 may be, for example, a general fisheye lens or a wide-angle lens. A wide-angle lens is a lens with a wide angle of view (a lens with a short focal length), for example, a lens with a focal length of 50 mm or less, preferably a lens with a focal length of 35 mm or less.

[0029] By configuring the optical system as described above, high resolution can be obtained in the high-resolution region, while in the low-resolution region, the increase amount of the image height y with respect to the semi-angle of view θ per unit is made small, and it becomes possible to image a wider angle of view. Therefore, while having a wide angle of view equivalent to that of a fisheye lens as the imaging range, a high resolution can be obtained in the high-resolution region.

[0030] In this embodiment, in the high-resolution region (low-distortion region), the projection characteristics approximate those of a central projection method (y = f × tanθ) or an equidistant projection method (y = f × θ), which are the projection characteristics of a general imaging optical system. Therefore, in the high-resolution region (low-distortion region), the optical distortion is small, and it is possible to generate a detailed image.

[0031] Next, with reference to FIG. 3, a method for measuring the height of the load 12 loaded on the truck 10 will be described. In this embodiment, it is necessary to measure the height of the load 12 in order to predict the possibility of a collision between the load 12 and a structure such as a bridge girder on the road. FIG. 3 is an explanatory diagram of a method for measuring the height of the load 12.

[0032] As shown in FIG. 3, the imaging unit 20 is attached above the driver's seat of the truck 10, and can capture the upper part of the load 12 within the imaging angle of view. Further, the upper part of the load 12 is included not in the first angle of view 30 which is the low-resolution region of the imaging unit 20, but in the second angle of view 31 (second angle of view region 31b) which is the high-resolution region. In other words, the imaging unit 20 is installed such that the upper part of the load 12 is included in the second angle of view 31 (second angle of view region 31b) which is the high-resolution region. That is, the imaging unit 20 is disposed above the truck 10 so as to form an image of the load 12 of the truck 10 in the second angle of view region 31b.

[0033] In this embodiment, first, the upper end portion 12t of the load is detected from the image (second angle of view region 31b of the captured image) acquired by the imaging unit 20. This can be detected using an object detection unit 503 described later. Subsequently, the load distance L1 to the upper end portion 12t of the load is measured. This can be measured using a distance measurement unit 504 described later.

[0034] Subsequently, it is confirmed at which angular position (angular region) of the image acquired by the imaging unit 20 the upper end portion 12t of the loaded object is present. In other words, it is confirmed at which pixel position in the vertical direction of the image the upper end portion 12t of the loaded object is present, and the angle θ1 is obtained. The angle θ1 can be calculated if the angle of the 0th pixel in the vertical direction of the image is grasped in advance. In the example of FIG. 3, it is approximately 0 degrees.

[0035] From the above results, the height H1 of the loaded object can be obtained by the following formula (2).

[0036] H1 = L1 * Sin(θ1) ···(2) Thereby, using the image acquired by the imaging unit 20, it is possible to obtain the height H1 of the loaded object with reference to the installation position of the imaging unit 20.

[0037] Next, with reference to FIG. 4, a method for measuring the height of the bridge girder of the viaduct (structure) 40 in front on the traveling road surface of the truck 10 will be described. In the present embodiment, in order to predict the possibility of collision between the loaded object 12 and a structure such as a bridge girder on the road, it is necessary to measure the height of the bridge girder of the viaduct 40. FIG. 4 is an explanatory diagram of a method for measuring the height of the bridge girder of the viaduct 40.

[0038] As shown in FIG. 4, the imaging unit 20 is attached above the driver's seat of the truck 10, and can capture the upper part of the viaduct 40 by including the upper part of the viaduct 40 in the imaging angle. Further, the upper part of the viaduct 40 is included not in the first angular region 30 which is a low-resolution region of the imaging unit 20, but in the second angular region 31 (the first angular region 31a) which is a high-resolution region. In other words, the imaging unit 20 is installed such that the upper part of the viaduct 40 is included in the second angular region 31 (the first angular region 31a) which is a high-resolution region. That is, the imaging unit 20 is arranged above the truck 10 so as to form an image of the viaduct 40 in front of the traveling direction of the truck 10 in the first angular region 31a.

[0039] In this embodiment, first, the bridge girder end 40t of the viaduct 40 is detected from the image (the first angular field region 31a of the captured image) acquired by the imaging unit 20. This can be detected using an object detection unit described later. Subsequently, the viaduct distance L2 to the bridge girder end 40t is measured. This can be measured using a distance measurement unit described later.

[0040] Subsequently, it is confirmed at which angular position in the image acquired by the imaging unit 20 the viaduct distance L2 exists, in other words, at which pixel position in the vertical direction of the image the bridge girder end 40t exists, and the angle θ2 is obtained. The angle θ2 can be calculated if the angle of the 0th pixel in the vertical direction of the image is grasped in advance, and is approximately 0 degrees in the example of FIG. 4.

[0041] From the above results, the bridge girder height H2 can be obtained by the following formula.

[0042] H2 = L2 * Sin(θ2) ···(3) Thereby, it is possible to obtain the bridge girder height H2 with respect to the installation position of the imaging unit 20 using the image acquired by the imaging unit 20.

[0043] Next, with reference to FIG. 5, the internal processing of the imaging device 100 in this embodiment will be described. FIG. 5 is a block diagram of the imaging device 100. The imaging device 100 includes an imaging unit 20, a control processing unit 21, a display unit 22, and a warning unit 506. Note that the imaging unit 20, the control processing unit 21, the display unit 22, and the warning unit 506 that constitute the imaging device 100 may be arranged separately from each other, or at least two of them may be integrally configured.

[0044] The imaging unit 20 includes an optical system 202 that forms an optical subject image (optical image) from external light, and an imaging element 210 that acquires an image obtained by imaging the subject. The optical system 202 is configured by combining a plurality of lenses and forms images with two different angles of view. As described with reference to FIGS. 2(A) and 2(B), the optical system 202 has an optical characteristic of forming a high-resolution optical image in a peripheral angle-of-view region away from the optical axis and forming a low-resolution optical image in a narrow angle-of-view region around the optical axis. The optical image formed by the optical system 202 is input to the imaging element 210.

[0045] The imaging element 210 is a photoelectric conversion element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The imaging element 210 has a light-receiving surface that is a photoelectric conversion area on the photoelectric conversion element, and photoelectrically converts the optical image received on the light-receiving surface into an electrical signal. The electrical signal converted by the imaging element is converted and processed into a predetermined image signal inside the imaging element 210 and output to the control processing unit 21.

[0046] Next, the control processing unit 21 will be described. The control processing unit 21 includes a video processing unit 501, a display processing unit 502, an object detection unit 503, a distance measurement unit 504, and a warning determination unit (comparison unit) 505.

[0047] The video processing unit 501 develops the image signal transmitted from the imaging unit 20 into a video and performs processes such as WDR (Wide Dynamic Range) correction, gamma correction, LUT processing, and distortion correction. By this processing, the image becomes easier to view when displayed on the display unit 22, and furthermore, the detection rate of the object detection process performed inside the object detection unit 503 that uses this image is improved. The image signal processed by the video processing unit 501 is input to the object detection unit 503 and the display processing unit 502.

[0048] The object detection unit 503 performs object detection processing using the image signal output from the video processing unit 501, and determines whether there are objects such as a loaded object or an elevated bridge (structure) in the image. Deep learning can be used for object detection. For example, as the deep learning, it is preferable to use YOLO (You Only Look Once) which is easy to learn and fast to detect. As the deep learning, SSD (Single Shot MultiBox Detector), Faster R-CNN (Regional Convolution Neural Network), Fast R-CNN, R-CNN, etc. may also be used. In addition, the training data applied to improve the accuracy of object detection may be divided according to the region (field of view) (different training data may be used according to the region). That is, the training data may be divided between the direction of the loading platform 11 behind the track 10 and the direction of the elevated bridge 40 in front (the training data for the rear and the training data for the front may be made different).

[0049] The object detection unit 503 performs semantic segmentation on the detected object, such as a loaded object or a structure in front of the elevated bridge, using deep learning. As a result, it becomes possible to accurately grasp the shape of the detected object. Then, for the loaded object 12, the end portion at the highest position among the detected shapes is detected, and the detected end portion is set as the upper end portion 12t of the loaded object as shown in FIG. 3. Also, for the elevated bridge 40, attention is paid to the portion (bridge girder portion) that hits the upper part at the time of passage of the track 10 among the detected shapes. For example, when the imaging unit 20 is installed at the center position in the vehicle width direction of the track 10, the image region at the left-right center position in the traveling direction is interpreted as the bridge girder portion and attention is paid. Then, among the shapes of the bridge girder portion of the detected elevated bridge 40, the lowest portion is set as the bridge girder end portion 40t as shown in FIG. 4.

[0050] The distance measurement unit 504 calculates (performs distance measurement) the distance from the installation position of the imaging unit 20 to the object detected by the object detection unit 503. As a distance measurement method, for example, there is a method of estimating the distance from deep learning. As an example, there is a method of calculating a distance value by analyzing information such as blurring of the image of the detected object by deep learning. As another method, the imaging unit 20 may be a stereo camera, and distance measurement may be performed using the principle of triangulation. Then, the distance measurement unit 504 uses the above-mentioned formulas (2) and (3) to calculate the height of the load on the loading platform 11 and the height of structures such as the bridge girder portion of the viaduct ahead.

[0051] The display processing unit 502 generates an image to be displayed on the display unit 22. The display processing unit 502 performs, for example, image cutting-out processing according to the display resolution of the display unit 22 based on the image signal received from the video processing unit 501 to generate a display image. Further, the display processing unit 502 may receive the object detection result from the object detection unit 503 and superimpose the result on the above-mentioned display image.

[0052] The warning determination unit 505 determines whether to give an instruction to issue a warning to the warning unit 506 using the distance measurement result by the distance measurement unit 504. Specifically, the warning determination unit 505 receives two distance measurement results, the height of the load (load height H1) and the bridge girder height H2 (height of the structure), from the distance measurement unit 504, and compares the load height H1 with the bridge girder height H2. When the load height H1 is higher than the bridge girder height H2 (H1 > H2), the warning determination unit 505 gives an instruction to issue a warning. On the other hand, when the load height H1 is lower than the bridge girder height H2 (H1 < H2), the warning determination unit 505 does not give an instruction to issue a warning.

[0053] The warning unit 506 receives an instruction to issue a warning from the warning determination unit 505 and issues a warning to the driver of the truck 10. That is, the warning unit 506 performs processing to issue a warning to the driver of the truck 10 based on the comparison result by the warning determination unit 505. By this warning, the driver of the truck 10 can be made aware in advance of the possibility of a collision between the elevated structure ahead on the road and the load on the loading platform 11. The warning unit 506 is, for example, the audio speaker of the display unit 22. However, the warning unit 506 is not limited to this, and a warning buzzer that notifies by sound or the like may be provided separately, or a warning unit that notifies by light or color such as a patrol lamp may be used.

[0054] Next, with reference to FIG. 6, an example of the control method executed by the control processing unit 21 in the present embodiment will be described. FIG. 6 is a flowchart showing the control method in the present embodiment. Each step in FIG. 6 is executed by a CPU (not shown) in the control processing unit 21 calling a program corresponding to the processing content from the ROM and expanding it in the RAM. During the execution of each step in FIG. 6, the truck 10 is in a traveling state, and each step is constantly repeated during traveling.

[0055] First, in step S101, the control processing unit 21 controls the imaging unit 20 attached to the upper part of the driver's seat of the truck 10, and particularly appropriately sets and controls the imaging element 210 to acquire imaging data (imaging image).

[0056] Subsequently, in step S102, the control processing unit 21 uses the object detection unit 503 to execute object detection processing in the forward direction on the traveling road surface of the truck 10. The angular direction of the angle to be detected is the region of the second angular range 31 that is a high-resolution region in the imaging unit 20.

[0057] Subsequently, in step S103, the control processing unit 21 determines whether a structure provided above the road such as an elevated bridge has been detected in the forward direction on the traveling road surface by the object detection processing in step S102. If no structure is detected, the process returns to step S101. On the other hand, if a structure is detected, the process proceeds to step S104.

[0058] In step S104, the control processing unit 21 estimates (performs distance measurement) the distance to the viaduct detected in step S102 using the distance measurement unit 504. The distance to the viaduct is estimated from the image captured by the imaging unit 20 using the aforementioned deep learning or the like.

[0059] Subsequently, in step S105, the control processing unit 21 calculates the height of the bridge girder portion of the viaduct detected in step S102 using the distance measurement unit 504. The height of the bridge girder portion of the viaduct is calculated using the aforementioned formula (3) and is the height information based on the installation position of the imaging unit 20.

[0060] Subsequently, in step S106, the control processing unit 21 performs object detection processing on the direction of the loading platform 11 of the truck 10 using the object detection unit 503. This is performed on the imaging data (captured image) acquired in step S101 and may be the same frame data as the data used in the object detection processing in step S102. Note that the angular direction of the image to be detected is the region of the second angular range 31 that is a high-resolution region in the imaging unit 20.

[0061] Subsequently, in step S107, the control processing unit 21 determines whether a loaded object is detected in the direction of the loading platform 11 on the truck 10 by the object detection processing in step S106. If no loaded object is detected, this flow ends. That is, when the control processing unit 21 does not detect a loaded object by the object detection unit 503, it does not perform a comparison between the height of the loaded object and the height of the viaduct (step S110 described later). On the other hand, if a loaded object is detected, it proceeds to step S108.

[0062] In step S108, the control processing unit 21 estimates (performs distance measurement) the distance to the loaded object detected in step S106 using the distance measurement unit 504. The distance to the loaded object is estimated from the captured image acquired by the imaging unit 20 using deep learning or the like as described above.

[0063] Subsequently, in step S109, the control processing unit 21 calculates the height of the loaded object detected in step S106 using the distance measurement unit 504. This is calculated using the aforementioned formula (2) and is the height information based on the installation position of the imaging unit 20.

[0064] Subsequently, in step S110, the control processing unit 21 compares the height of the viaduct calculated in step S105 with the height of the loaded object calculated in step S109 using the warning determination unit 505. Since both the height of the viaduct and the height of the loaded object are calculated based on the installation position of the imaging unit 20, a relative height comparison is possible. If the height of the viaduct is higher than the height of the loaded object, the warning determination unit 505 determines that warning issuance is unnecessary, and this flow ends. On the other hand, if the height of the loaded object is higher than the height of the viaduct, the warning determination unit 505 determines that warning issuance is necessary and proceeds to step S111.

[0065] In step S111, the control processing unit 21 controls the warning unit 506 so that the warning unit 506 actually issues a warning. This is, for example, the voice speaker of the display unit 22, which conveys a warning to the driver by voice. As a result, the driver can be aware in advance of the possibility of a collision between the viaduct structure ahead on the road and the loaded object on the loading platform 11.

[0066] This flowchart is repeatedly executed. Therefore, the truck 10 gradually approaches the detected viaduct while driving, and the calculation accuracy of the height improves as it gets closer.

[0067] Also, this flowchart performs the detection of the loaded object and the calculation of the height after detecting the viaduct. Therefore, even if the loaded object moves due to vibration and its height changes during the driving of the truck 10 before the viaduct is detected, it is possible to calculate the height of the changed loaded object at the time of viaduct detection.

[0068] According to this embodiment, the truck 10 can perform both the monitoring process of the vehicle's load and the monitoring process of the front in the traveling direction using only one camera (imaging unit 20). Therefore, even when there is a load on the vehicle, it is possible to appropriately warn of the possibility of a collision. Thus, the configuration of this embodiment can be realized with a simple configuration and low cost compared to the case of using a plurality of cameras.

[0069] <Second Embodiment> Next, a second embodiment of the present invention will be described. In the first embodiment, a method of warning the driver of the possibility of a collision by calculating the height of the load during the running of the truck and comparing the height of the load with the height of a structure such as an overpass detected ahead has been described. On the other hand, in this embodiment, a case where the driver confirms the height of the load before the start of the truck's travel will be described. Thereby, it is possible to prevent an erroneous determination of a warning due to a misrecognition of the height of the load. Note that each configuration and each process such as the imaging unit 20 and the control processing unit 21 are the same as those in FIGS. 1 to 5 referred to in the first embodiment.

[0070] Referring to FIG. 7, an example of the process (control method) before the start of travel when the engine of the truck 10 in this embodiment is started will be described. FIG. 7 is a flowchart showing an example of the control method in this embodiment. Each step in FIG. 7 is executed by a CPU (not shown) in the control processing unit 21 calling a program corresponding to the processing content from the ROM and expanding it in the RAM.

[0071] First, in step S201, the engine of the truck 10 is started. The engine of the truck 10 is started by detecting that the engine switch has been pressed by the driver. As a result, each electrical and electronic component inside the truck 10 including the imaging unit 20 and the control processing unit 21 is energized and started. Note that at this point, the truck 10 has not yet started traveling.

[0072] Subsequently, in step S202, the control processing unit 21 controls the imaging unit 20 attached to the upper part of the driver's seat of the truck 10, and acquires imaging data (imaging image) by appropriately setting and controlling the imaging element 210 in particular.

[0073] Subsequently, in step S203, the control processing unit 21 uses the object detection unit 503 to perform object detection processing in the direction of the loading platform 11 of the truck 10. Note that the angular range to be detected is the region of the second angular range 31 that is a high-resolution region in the imaging unit 20.

[0074] Subsequently, in step S204, the control processing unit 21 determines whether or not a loaded object has been detected in the direction of the loading platform 11 on the truck 10 by the object detection processing in step S203. If no loaded object is detected, the process proceeds to step S205. On the other hand, if a loaded object is detected, the process proceeds to step S206.

[0075] In step S205, the control processing unit 21 notifies the driver of the truck 10 that no loaded object has been detected. This may be notified, for example, by displaying a message on the display unit 22, or by using the voice function of the display unit 22. Thereby, the driver can grasp in advance that there is no loaded object at a position higher than the position of the imaging unit 20 (that is, the original height of the truck 10). Also, if the detection of the loaded object fails due to dirt on the imaging unit 20 or the like, if the driver himself / herself is aware of loading a high loaded object, it is possible to notice the failure of detection before starting to drive. Note that when no loaded object is detected, it is not necessary to execute the flowchart of FIG. 6 of the first embodiment after the truck 10 starts to run.

[0076] In step S206, the control processing unit 21 uses the display processing unit 502 to display an image of the loaded object. For example, the display processing unit 502 cuts out the angular range in which the loaded object is reflected from the imaging data acquired by the imaging unit 20 and displays it on the display unit 22.

[0077] Subsequently, in step S207, the control processing unit 21 uses the object detection unit 503 and the display processing unit 502 to detect the highest position of the loaded cargo and displays a request for confirmation to the driver of the truck 10. Similar to the first embodiment, the highest position of the loaded cargo is detected as the upper end portion 12t of the loaded cargo by the object detection unit 503 using deep learning.

[0078] Here, with reference to FIG. 8, an example of the display in step S207 will be described. FIG. 8 is an example of the display screen (loaded cargo confirmation screen) of the display unit 22 in the present embodiment. In the present embodiment, the display unit 22 is a confirmation means for requesting the driver of the vehicle to confirm the height of the loaded cargo 12 measured by the distance measurement unit 504 based on the image captured by the imaging device 210 before the start of the vehicle's travel.

[0079] On the display screen of the display unit 22, the loaded cargo 12 is displayed, and a frame of the loaded cargo end portion 801 is displayed so as to surround the detected upper end portion 12t of the loaded cargo. For the displayed frame, a text (confirmation display 802) requesting the driver to confirm whether that part is surely the upper end portion of the loaded cargo 12 is displayed. When the position of the loaded cargo end portion 801 is surely the upper end portion of the loaded cargo, the driver of the truck 10 presses the Yes button displayed on the display unit 22. On the other hand, when the loaded cargo end portion 801 is not the upper end portion of the loaded cargo, the driver presses the No button displayed on the display unit 22. In the present embodiment, the display unit 22 is a touch panel, and the control processing unit 21 can detect which of the Yes or No buttons has been pressed. In the present embodiment, the loaded cargo end portion 801 and the confirmation display 802 are drawn by the display processing unit 502 and displayed on the display unit 22.

[0080] Subsequently, in step S208 of FIG. 7, the control processing unit 21 determines whether the Yes button in FIG. 8 has been pressed, that is, which of the Yes button and the No button has been pressed. If it is determined that the Yes button has been pressed, the process proceeds to step S210. On the other hand, if it is determined that the No button has been pressed, the process proceeds to step S209.

[0081] In step S209, the control processing unit 21 determines that the upper end portion 12t of the load detected by the object detection unit 503 is not the upper end of the load, that is, it is determined that the upper end portion 12t of the load is erroneously detected. Therefore, the control processing unit 21 prompts the driver to manually input the height of the load, and sets the height of the load according to the input content. For example, when the upper end of the load is not within the imaging range of the imaging unit 20 (when it is hidden in a blind spot), or when the cause is considered to be the imaging unit 20 itself. In this case, the driver uses an input device such as the touch panel of the display unit 22 to input the load height H1, and the control processing unit 21 uses the input value for comparison with the bridge height H2. That is, after the start of the travel of the truck 10, a flowchart similar to FIG. 6 referred to in the first embodiment is executed, but the load height information used in step S110 at that time is the input value by the driver. Also, when the upper end of the load is hidden in the blind spot of the imaging unit 20 and cannot be detected, the control processing unit 21 may perform an operation such as shifting the position of the imaging unit 20 in the vehicle width direction of the truck 10, and try to detect the upper end portion of the load again. That is, when at least a part of the load is not detected by the object detection unit 503, the warning determination unit 505 may move the imaging unit 20 in the vehicle width direction of the truck 10.

[0082] In step S210, the control processing unit 21 determines that the upper end portion 12t of the load detected by the object detection unit 503 has been correctly detected. In this case, after the start of the travel of the truck 10, a flowchart similar to FIG. 6 referred to in the first embodiment is executed, and the load height information used in step S110 is the value calculated in step S109.

[0083] According to this embodiment, even when the control processing unit 21 misrecognizes the position of the upper end portion of the load, the driver can grasp the misrecognition before the start of the travel of the truck 10. Therefore, by measures such as the driver's manual input of the height, it is possible to reduce the possibility of collision between the load and structures such as elevated bridges.

[0084] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be realized by a circuit (for example, ASIC) that implements one or more functions.

[0085] According to each embodiment, it is possible to perform both monitoring processes of monitoring the load on the vehicle and monitoring the front part in the traveling direction of the vehicle with one camera, and it is possible to appropriately warn of the possibility of collision even when there is a load on the vehicle. Therefore, according to each embodiment, it is possible to provide an imaging device, a control method for the imaging device, a program, etc., which can appropriately warn of the possibility of collision even when there is a load on the vehicle with a simple configuration.

[0086] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) An imaging device disposed on a vehicle, An imaging unit disposed above the vehicle so as to form an image of a structure in front of the vehicle in a traveling direction in a first angle-of-view region and an image of a load on the vehicle in a second angle-of-view region; A distance measuring unit that measures the height of the structure using the first angle-of-view region and measures the height of the load on the vehicle using the second angle-of-view region; A comparison unit that compares the height of the structure and the height of the load using the distance measurement result by the distance measuring unit; A warning unit that performs processing to warn a driver of the vehicle based on a comparison result by the comparison unit. An imaging device, characterized by comprising the above. (Configuration 2) The imaging unit has an optical system, The optical system is a wide-angle lens. The imaging device according to claim 1, characterized by the above. (Configuration 3) The imaging unit has an optical system, The optical system is a fish-eye lens. The imaging device according to claim 1, characterized in that... (Configuration 4) The imaging unit has an optical system that forms a low-resolution region on the central side of the light-receiving surface and a high-resolution region on the peripheral side of the light-receiving surface. The first angular field region and the second angular field region are each the high-resolution region. The imaging device according to any one of claims 1 to 3, characterized in that... (Configuration 5) The distance measuring unit performs distance measurement of the load after the structure is detected. The imaging device according to any one of claims 1 to 4, characterized in that... (Configuration 6) It further has a confirmation means for requesting the driver of the vehicle to confirm the height of the load measured by the distance measuring unit based on an image captured by the imaging unit before the vehicle starts running. The imaging device according to any one of claims 1 to 5, characterized in that... (Configuration 7) It further has an object detection unit capable of detecting the load and the structure. When the load is not detected by the object detection unit, the comparison unit does not perform the comparison. The imaging device according to any one of claims 1 to 6, characterized in that... (Configuration 8) It further has an object detection unit capable of detecting the load and the structure. When at least a part of the load is not detected by the object detection unit, the comparison unit moves the imaging unit in the vehicle width direction. The imaging device according to any one of claims 1 to 7, characterized in that... (Method 1) A control method for an imaging device disposed in a vehicle, comprising: An imaging step of using an imaging unit disposed in the vehicle to form an image of a structure in front of the traveling direction of the vehicle in a first angular field region and an image of a load of the vehicle in a second angular field region. A distance measurement step of measuring the height of the structure using the first angular field of view area and measuring the height of the load of the vehicle using the second angular field of view area; A comparison step of comparing the height of the structure and the height of the load using the distance measurement result in the distance measurement step; A warning step of performing processing to give a warning to the driver of the vehicle based on the comparison result in the comparison step; A control method for an imaging device, characterized by comprising the above. (Configuration 9) A program for causing a computer to execute a control method for an imaging device disposed on a vehicle, The control method includes: An imaging step of forming an image of a structure in front of the traveling direction of the vehicle in a first angular field of view area using an imaging unit disposed in the vehicle information, and forming an image of the load of the vehicle in a second angular field of view area; A distance measurement step of measuring the height of the structure using the first angular field of view area and measuring the height of the load of the vehicle using the second angular field of view area; A comparison step of comparing the height of the structure and the height of the load using the distance measurement result in the distance measurement step; A warning step of performing processing to give a warning to the driver of the vehicle based on the comparison result in the comparison step; A program, characterized by comprising the above.

[0087] The preferred embodiments of the present invention have been described above. However, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist. Some of the above-described embodiments may be appropriately combined.

Explanation of Reference Numerals

[0088] 10 Truck (vehicle) 12 Load 20 Imaging unit 40 Overpass (structure) 100 Imaging device 504 Distance measurement unit 505 Warning determination unit (comparison unit) 506 Warning section

Claims

1. An imaging device disposed in a vehicle, comprising: an imaging unit disposed above the vehicle so as to form an image of a structure in front of the vehicle in a first angle-of-view region and an image of a load of the vehicle in a second angle-of-view region; a distance measuring unit that measures the height of the structure using the first angle-of-view region and measures the height of the load of the vehicle using the second angle-of-view region; a comparison unit that compares the height of the structure and the height of the load using the distance measurement result by the distance measuring unit; a warning unit that performs processing to give a warning to a driver of the vehicle based on a comparison result by the comparison unit. An imaging device, characterized by comprising the above.

2. The imaging unit has an optical system, wherein the optical system is a wide-angle lens. The imaging device according to claim 1, characterized by the above.

3. The imaging unit has an optical system, wherein the optical system is a fish-eye lens. The imaging device according to claim 1, characterized by the above.

4. The imaging unit has an optical system that forms a low-resolution region on a central side of a light-receiving surface and forms a high-resolution region on a peripheral side of the light-receiving surface, wherein each of the first angle-of-view region and the second angle-of-view region is the high-resolution region. The imaging device according to any one of claims 1 to 3, characterized by the above.

5. The distance measuring unit performs distance measurement of the load after the structure is detected. The imaging device according to any one of claims 1 to 3, characterized by the above.

6. The imaging device further includes a confirmation unit that requests a driver of the vehicle to confirm the height of the load measured by the distance measuring unit based on an image captured by the imaging unit before the vehicle starts running. The imaging device according to any one of claims 1 to 3, characterized by the above.

7. The imaging device further includes an object detection unit capable of detecting the load and the structure, wherein when the load is not detected by the object detection unit, the comparison unit does not perform the comparison. The imaging device according to any one of claims 1 to 3, characterized by the above.

8. The imaging device further includes an object detection unit capable of detecting the load and the structure, wherein when at least a part of the load is not detected by the object detection unit, the comparison unit moves the imaging unit in a vehicle width direction of the vehicle. The imaging device according to any one of claims 1 to 3, characterized by the above.

9. A control method for an imaging device disposed in a vehicle, comprising: An imaging step of forming an image of a structure in front of the traveling direction of the vehicle in a first angular field of view region and forming an image of the load of the vehicle in a second angular field of view region by using an imaging unit disposed in the information of the vehicle; A distance measuring step of measuring the height of the structure using the first angular field of view region and measuring the height of the load of the vehicle using the second angular field of view region; A comparison step of comparing the height of the structure and the height of the load using the distance measurement result in the distance measuring step; A warning step of performing processing to give a warning to the driver of the vehicle based on the comparison result in the comparison step; A control method for an imaging device, characterized by comprising the above steps.

10. A program for causing a computer to execute a control method for an imaging device disposed in a vehicle, wherein the control method comprises: An imaging step of forming an image of a structure in front of the traveling direction of the vehicle in a first angular field of view region and forming an image of the load of the vehicle in a second angular field of view region by using an imaging unit disposed in the information of the vehicle; A distance measuring step of measuring the height of the structure using the first angular field of view region and measuring the height of the load of the vehicle using the second angular field of view region; A comparison step of comparing the height of the structure and the height of the load using the distance measurement result in the distance measuring step; A warning step of performing processing to give a warning to the driver of the vehicle based on the comparison result in the comparison step; A program, characterized by comprising the above steps.

Images