Imaging system and movable body

Abstract

To pick up images of a plurality of imaging areas favorably and simultaneously with one imaging apparatus, in an imaging system mounted on a movable body.SOLUTION: An imaging system comprises: an imaging unit 107 that is mounted on a movable body; and a control unit 108 that outputs image data acquired by the imaging unit on a display 110. The imaging unit can perform imaging at a higher resolution in a peripheral side area 20b than a center side area 20c of the imaging angle of view. In one part of the peripheral side area, the imaging unit picks up an image of a first imaging area in the direction of travel of the movable body, and in the other part of the peripheral side area, picks up an image of a third imaging area that is closer to the movable body than a second imaging area where the imaging unit picks up an image in the center side area.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an imaging system mounted on a moving body such as a work vehicle. [Background technology]

[0002] Some work vehicles used on construction sites are equipped with a system that uses sensors and cameras attached to the vehicle to monitor the intrusion of workers into the area around the vehicle and issues a warning when intrusion is detected. On the other hand, a certain level of quality is required for the work performed by the work vehicle. For example, bulldozers are required to have the quality of excavation and leveling, and road rollers are required to have the quality of compaction.

[0003] Patent Document 1 discloses a system that displays a composite image obtained by capturing images of a work area around a work vehicle and a nearby area that comes into contact with the work vehicle using multiple cameras attached to the work vehicle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-195008 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the system disclosed in Patent Document 1 requires the preparation of multiple cameras. In addition, since these cameras are attached to the roof of the work vehicle, the resolution of the cameras may be insufficient to obtain detailed information about the work area of ​​the work vehicle (for example, the presence of small foreign objects or debris in the work area). Furthermore, the imaging angle of view of the cameras must be narrowed to obtain detailed information about the work area, and the imaging angle of view may be insufficient to confirm the safety of the work vehicle.

[0006] The present invention provides an imaging system that enables a single camera to simultaneously and satisfactorily capture images of a plurality of imaging regions. [Means for solving the problem]

[0007] An imaging system for a moving body according to one aspect of the present invention includes an imaging unit arranged on the moving body, and a control unit that outputs image data acquired by the imaging unit to a display unit. The imaging unit has a configuration that allows imaging with a higher resolution in a peripheral region of an imaging angle of view than in a central region. The imaging unit is characterized in that it images a first imaging region in a traveling direction of the moving body in a part of the peripheral region, and images a third imaging region closer to the moving body than the second imaging region imaged in the central region in another part of the peripheral region. Note that a moving body having the imaging system also constitutes another aspect of the present invention. Effect of the Invention

[0008] According to the present invention, imaging of the first, second and third imaging regions can be performed simultaneously by one imaging section, and high-resolution imaging of the first and third imaging regions can be performed. [Brief description of the drawings]

[0009] [Figure 1] 1 is an external view of a work vehicle equipped with an imaging device according to an embodiment; [Diagram 2] FIG. 4 is a diagram showing optical characteristics of the imaging apparatus according to the embodiment. [Diagram 3] 3A and 3B are diagrams showing the position and imaging direction of an imaging unit of the imaging device according to the embodiment. [Figure 4] FIG. 2 is a block diagram showing the configuration of an imaging unit and a control unit in the embodiment. [Diagram 5] 4 is a flowchart showing a process according to an embodiment. [Figure 6] 10A to 10C are diagrams showing examples of captured images in the embodiment. [Figure 7] FIG. 4 is a diagram showing an example of a display image in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0011] 1 shows the appearance of a road roller (work vehicle) 100, which is an example of a mobile body on which an imaging system for a mobile body according to an embodiment is mounted. The road roller 100 is a combined road roller equipped with one iron wheel on one of the front and rear (the front in the figure) and two tires on the other (the rear in the figure), and performs work of compacting the ground with the iron wheels while traveling.

[0012] The road roller may be a tandem road roller with one iron wheel at the front and one at the back, a macadam road roller with one iron wheel at the front and two at the rear, or a combined road roller. The work vehicle may be an asphalt finisher, a motor grader, a bulldozer, or another vehicle other than a road roller.

[0013] The rolling roller 101 as an iron wheel is, for example, a metal drum. The rolling roller 101 is rotated by a hydraulic motor (not shown). The tire 102 is, for example, a rubber tire, and is rotated by a hydraulic motor (not shown).

[0014] In the drive unit 111, a hydraulic pump (not shown) is driven by a drive source such as an engine, and a hydraulic motor is driven by pressure oil delivered from the hydraulic pump, thereby driving the rolling roller 101 and the tire 102 to rotate.

[0015] The driver's seat 105 has a seat where an operator who operates the road roller 100 sits. The upper part of the driver's seat 105 is covered by a roof section 106. The control platform 103 is provided in front of the driver's seat 105, and supports a steering device 104 and a display section 110. The operator can control the traveling direction of the road roller 100 (i.e., steer the road roller 100) by operating the steering device 104.

[0016] An imaging unit 107 is disposed in front of the road roller 100 as an imaging device included in the imaging system for a mobile body. The imaging unit 107 images the front, which is the traveling direction (traveling direction) of the road roller 100, and the area below including the vicinity of the road roller 100, to generate imaging data. In addition to the imaging unit 107, the imaging system for a mobile body includes a control unit 108, an alarm unit (alarm means) 109, and a display unit 110.

[0017] The control unit 108 includes a microcomputer consisting of one or more processors such as a CPU (Central Processing Unit) that performs calculations and control, and a ROM (Read Only Memory) and a RAM (Random Access Memory) that are main storage devices. The ROM stores basic setting data and programs for executing processes related to the camera. The microcomputer calls up a program corresponding to the process to be executed from the ROM, loads it into the RAM, and causes each block in the control unit 108, which will be described later, to execute its operation.

[0018] Furthermore, the control unit 108 is connected to the imaging unit 107, the display unit 110, and the drive unit 111 via cables (not shown). The control unit 108 executes various processes on the imaging data acquired from the imaging unit 107, and transmits the processing results to the display unit 110. That is, the control unit 108 outputs image data generated by executing various processes on the imaging data acquired from the imaging unit 107 to the display unit 110. Furthermore, the control unit 108 controls the alarm unit 109 and the drive unit 111 according to the processing results.

[0019] The alarm unit 109 has a warning light such as a rotating light. The warning light may be configured with a multi-color LED, and the emitted color may be controlled depending on the type of alarm. The alarm unit 109 may also be a buzzer that emits an alarm sound, and may be controlled to make the alarm sound different depending on the type of alarm.

[0020] The display unit 110 has a display device such as a liquid crystal display, and displays images to assist the operator in their work (for example, images of the front and bottom of the road roller 100). The operator seated in the driver's seat 105 can operate the steering device 104 while looking at the image displayed on the display unit 110 to steer the road roller 100.

[0021] Next, the optical characteristics of the optical system of the imaging unit 107 will be described. The imaging unit 107 of this embodiment has a configuration that can capture images with a higher resolution in the peripheral region than in the central region of the imaging angle of view. Specifically, the optical characteristics of the optical system are set so that images can be captured with a higher resolution in the peripheral region than in the central region of the imaging angle of view.

[0022] FIG. 2(A) shows the image height on the image plane (light receiving surface of the image sensor) for each angle of view of the optical system of this embodiment using contour lines. FIG. 2(B) shows the relationship (projection characteristic) between the half angle of view θ and the image height y of the optical system of this embodiment. In FIG. 2(B), the horizontal axis is the half angle of view (angle between the optical axis of the optical system and the incident light) θ (deg), and the vertical axis is the image height y (mm) on the light receiving surface. The dashed line in FIG. 2(B) shows the projection characteristic of a general equidistant projection (y=fθ), and the two-dot chain line shows the projection characteristic of a stereographic projection (y=2ftan(θ / 2)). The solid line shows the projection characteristic y(θ) of the optical system of this embodiment.

[0023] As shown in FIG. 2B, the optical system of this embodiment has a projection characteristic in which the increase rate of y with respect to θ changes from the optical axis center (θ=0) to the maximum half angle of view θmax, unlike the equidistant projection in which the image height y increases in proportion to the half angle of view θ. Specifically, between the optical axis center (θ=0) and the maximum half angle of view θmax, y is lower than y=fθ, and the increase rate of y with respect to θ increases as θ increases. In other words, the increase rate of y with respect to θ in the peripheral region away from the optical axis is larger than the increase rate of y with respect to θ in the central region near the optical axis. Such a projection characteristic is similar to the stereoscopic projection. However, the optical system of this embodiment has a projection characteristic in which the increase rate of y with respect to θ in the central region is smaller than that in the stereoscopic projection, and the increase rate of y with respect to θ in the peripheral region is larger.

[0024] Furthermore, when the amount of change in image height y per unit angle of view (i.e., the number of pixels of the image sensor per unit angle of view) is referred to as resolution, the optical system of this embodiment has a projection characteristic in which the resolution changes according to the half angle of view θ. The local resolution is expressed as the differential value dy(θ) / dθ of the projection characteristic y(θ) at the half angle of view θ. That is, the greater the slope of the projection characteristic y(θ) in FIG. 2(B), the higher the resolution. Also, the greater the interval between the contour lines in FIG. 2(A), the higher the resolution. In the optical system of this embodiment, the resolution in the peripheral region is higher than the resolution in the central region.

[0025] FIG. 2C shows the relationship between the half angle of view θ of the optical system and the resolution (resolution characteristic). In FIG. 2C, the horizontal axis is the half angle of view θ, and the vertical axis is the resolution (pix / deg). The dashed line in FIG. 2C shows the resolution characteristic of equidistant projection, and the two-dot chain line shows the resolution characteristic of stereoscopic projection. The solid line shows the resolution characteristic of the optical system of this embodiment. In equidistant projection, the resolution is the same regardless of the angle of view. In contrast, the optical system of this embodiment has a resolution characteristic in which the resolution in the central region is lower than y=fθ, the resolution in the peripheral region is higher than y=fθ, and the larger θ is, the higher the resolution becomes. Such a resolution characteristic is similar to that of stereoscopic projection. However, the optical system of this embodiment has a resolution characteristic in which the resolution in the central region is lower and the resolution in the peripheral region is higher than that of stereoscopic projection.

[0026] In the following description, as shown in Fig. 2(A) to (C), the range of the imaging angle of view from θ=0 to θmax is referred to as the full angle of view range 20a, and the central region of the full angle of view range 20a where the half angle of view θ is less than a predetermined half angle of view θa is referred to as the low resolution region 20c. Also, the peripheral region of the full angle of view range 20a where the half angle of view θ is equal to or greater than the predetermined half angle of view θa is referred to as the high resolution region 20b. The predetermined half angle of view θa here is θ when dy(θ) / dθ matches dy(θ) / dθ of y=fθ. With regard to distortion, the low resolution region 20c is a high distortion region with large distortion, and the high resolution region 20b is a low distortion region with smaller distortion than the high distortion region.

[0027] 2(A) to (C) are merely examples, and it is sufficient that a low-resolution region is formed on the central side and a high-resolution region is formed on the peripheral side. In this embodiment, the low-resolution region 20c and the high-resolution region 20b of the optical system of the imaging unit 107 are formed concentrically, but they do not necessarily have to be concentric. For example, the low-resolution region and the high-resolution region may have a distorted shape. In addition, the centers of gravity of the low-resolution region and the high-resolution region do not have to coincide with each other, and furthermore, each may be shifted from the center of the light receiving surface.

[0028] The optical system of this embodiment has optical characteristics that satisfy the following equation (1), where f is the focal length, θ is the half angle of view, y is the image height on the image plane, y(θ) is the projection characteristic, and θmax is the maximum half angle of view.

[0029] 0.20≦2ftan(θmax / 2) / y(θmax) ≦0.95 (1) It is more preferable that the numerical range of formula (1) is as follows:

[0030] 0.25≦2ftan(θmax / 2) / y(θmax)≦0.94 (1a) Moreover, it is more preferable that the numerical range of the formula (1) is as follows:

[0031] 0.30≦2ftan(θmax / 2) / y(θmax)≦0.80 (1b) In an optical system with such optical characteristics, the magnification in the radial direction from the optical axis can be adjusted by adjusting the projection characteristic y(θ). This makes it possible to control the aspect ratio in the radial direction from the optical axis and in the circumferential direction. As a result, unlike conventional fisheye lenses, it is possible to obtain a high-resolution optical image with a wide angle of view and little distortion on the periphery.

[0032] Next, the arrangement of the imaging unit 107 and the setting of the imaging area (imaging method) will be described with reference to Fig. 3. Fig. 3 shows an enlarged view of the imaging unit 107 and its surroundings in Fig. 1. In Fig. 3, the imaging unit 107 is arranged in front of the compaction roller 101 of the road roller 100. The imaging unit 107 is also arranged so that the optical axis of the optical system faces downward from the horizontal direction. The imaging unit 107 has a total angle of view of, for example, about 150°.

[0033] The imaging unit 107 is disposed so that an upper angle of view area of ​​the high resolution area 20b faces the traveling direction (forward) of the road roller 100, and a lower angle of view area of ​​the high resolution area 20b faces an imaging area (hereinafter referred to as a work area) near the contact position of the compaction roller 101 as a roller part on the ground. As a result, the low resolution area 20c faces an imaging area (hereinafter referred to as a front ground area) forward of the contact position of the compaction roller 101 as a roller part on the ground. That is, the imaging unit 107 images a first imaging area in front with a part of the high resolution area 20b, images a front ground area (second imaging area) with the low resolution area 20c, and simultaneously images a work area (third imaging area) closer to the work vehicle than the front ground area with another part of the high resolution area 20b.

[0034] FIG. 6 shows an example of an image generated from the imaging data from the imaging unit 107. An image 601 is one image including an image in the travel direction, an image of the ground area ahead, and an image of the work area. FIG. 6 shows triangular cones 602a and 602b as obstacles present in the travel direction. These triangular cones 602a and 602b are distant from the road roller 100, but are imaged at high resolution through the high resolution area (upper angle of view area) 20b of the imaging unit 107. FIG. 6 also shows fallen leaves 604 and rocks 603 as foreign objects present in the work area. These fallen leaves 604 and rocks 603 are small objects, but are imaged at high resolution through the high resolution area (lower angle of view area) 20b of the imaging unit 107. The obstacle corresponds to a first object related to safety, and the foreign object corresponds to a second object related to work quality (i.e., different in type from the first object).

[0035] In this way, by using the imaging unit 107 capable of capturing images at a higher resolution in the peripheral region (20b) of the entire angle of view range 20a than in the central region (20b), it is possible to clearly capture images of obstacles that may be contacted in the traveling direction and foreign objects that may lead to a decrease in work quality in the work area. Therefore, it is possible to more accurately detect such obstacles and foreign objects based on image data obtained by capturing images.

[0036] A general wide-angle camera or a fish-eye camera may be used as the imaging unit 107. In this case, a high-resolution image on the periphery side as shown in Fig. 6 cannot be obtained, but such a camera may be used as long as it can satisfactorily detect obstacles and foreign objects on the periphery side.

[0037] 4 shows detailed configurations of the imaging unit 107 and the control unit 108. The imaging unit 107 has an optical system 401 that forms a subject image (optical image) by focusing light from the outside, and an imaging element 402 that performs photoelectric conversion on the subject image to capture the image. The optical system 401 is configured by combining a plurality of lenses, and has the optical characteristics shown in FIGS. 2(A), (B), and (C). The imaging element 402 is a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor, and generates imaging data by photoelectrically converting the subject image on its light receiving surface (imaging surface). The imaging data read from the imaging element 402 is output to the control unit 108 at the subsequent stage.

[0038] The control unit 108 has a development processing unit 403, a distortion correction unit 404, an object detection unit 405, a display processing unit 406, and a stop determination unit 407. The development processing unit 403 performs development processing on imaging data from the imaging unit 107 (image sensor 402) to generate image data, and further performs image processing such as WDR (Wide Dynamic Range) correction, gamma correction, and LUT (Look Up Table) processing on the image data.

[0039] The distortion correction unit 404 performs distortion correction processing on all or part of the image data input from the development processing unit 403 as coordinate conversion for display on the rectangular screen of the display unit 110. At this time, if the input image data has little distortion, it is output directly to the display unit 110 without performing distortion correction processing. The processing by the development processing unit 403 and the distortion correction unit 404 can improve the visibility of the image displayed on the display unit 110 and can also improve the detection rate of the object detection processing performed by the object detection unit 405. The image data processed by the distortion correction unit 404 is output to the object detection unit 405 and the display processing unit 406.

[0040] The object detection unit 405 performs object detection processing using the image data input from the distortion correction unit 404, and determines whether or not obstacles such as vehicles, people, and traffic cones, and foreign objects such as fallen leaves and rocks, are present in the image data. Deep learning is used for the object detection processing. As another deep learning, for example, it is preferable to use YOLO (You Only Look Once), which is easy to learn and has fast detection. As another deep learning, SSD (Single Shot MultiBox Detector), Faster R-CNN (Regional Convolution Neural Network), Fast R-CNN, R-CNN, etc. may be used. The result of the object detection processing is output as four-point coordinate information, which is the four vertex coordinates of a rectangle (bounding box) indicating the position where the object is detected, and object name information indicating the classification of the detected object.

[0041] In addition, in the object detection process, different learning results may be applied according to the angle of view of the image data to detect different objects. That is, learning results of obstacles such as vehicles, people, and traffic cones may be applied to the upper part of the angle of view, and learning results of foreign objects such as fallen leaves and rocks may be applied to the lower part of the angle of view. In this way, for image data obtained through the imaging unit 107 arranged as described in FIG. 3, obstacle detection processing is performed in the traveling direction of the road roller 100, and foreign object detection processing is performed in the direction of the ground where compaction work is performed by the compaction roller 101 (work area). That is, by limiting the detection target depending on the angle of view, accurate and high-speed object detection processing can be realized.

[0042] The display processing unit 406 performs a cutout process or the like on the image data input from the distortion correction unit 404 in accordance with the display resolution of the display unit 110, and generates display image data to be displayed on the display unit 110. The display processing unit 406 may cut out a plurality of image regions from the input image data and synthesize them to generate one piece of display image data. For example, an image region in the traveling direction and an image region in the ground direction may be cut out and synthesized to generate one piece of display image data. As a result, a display image corresponding to the image data input to the display processing unit 406 is displayed on the display unit 110.

[0043] Furthermore, the display processing unit 406 adds the object detection result received from the object detection unit 405 to the display image data described above. For example, a bounding box is superimposed on the display image data to allow the observer, who is the operator, to clearly recognize the detected object and its position. As a result, a bounding box surrounding the object is superimposed on the display image on the display unit 110. Note that instead of displaying a bounding box, the outline of the object may be displayed in a bold line in a conspicuous color, or an arrow pointing to the object may be displayed.

[0044] Fig. 7 shows an example of an image (moving image) displayed on the display unit 110. The display image shown in Fig. 7 is divided into two parts, upper and lower, with the upper part including an image in the traveling direction (forward image) and the lower part including an image of the working area in the ground direction (working area image). This display image is generated by combining the image areas in the traveling direction and the ground direction cut out by the display processing unit 406 from the image data whose distortion has been corrected by the distortion correction unit 404 so as to connect them vertically.

[0045] In addition, a bounding box is also superimposed on the display image as a result of object detection by the object detection unit 405. Specifically, an obstacle detection frame 702a, which is a bounding box, is displayed in the forward image as a result of detecting the triangular cone 602a, and an obstacle detection frame 702b is displayed as a result of detecting the triangular cone 602b. By displaying the obstacle detection frames in this way, it becomes easier for the operator looking at the display unit 110 to notice the approach of an obstacle in the traveling direction.

[0046] Furthermore, if the road roller 100 is automated (unmanned) or remotely operated, an operator at a remote location can easily notice that the road roller is approaching an obstacle by looking at the display device at hand.

[0047] Furthermore, within the work area image, a foreign object detection frame 703, which is a bounding box, is displayed as a result of detecting rocks 603, and a foreign object detection frame 704 is displayed as a result of detecting fallen leaves 604. By displaying the foreign object detection frame in this way, even if the work area on the ground is in a blind spot for the operator sitting in the cab 105 while riding on the road roller 100, the presence of a foreign object in the work area can be easily noticed through the display unit 110. Also, if the road roller 100 is automated or remotely operated, an operator at a remote location can easily notice the presence of a foreign object near the compaction roller 101 by looking at the display device at hand.

[0048] The displayed obstacle detection frame and foreign object detection frame may be different in color or line type (including thickness) so that the operator can immediately distinguish between an obstacle and a foreign object.

[0049] By displaying images as described above, it is possible to prevent a decrease in safety due to contact with an obstacle by the road roller 100, and a decrease in work quality due to compaction by foreign objects.

[0050] The stop determination unit 407 determines whether or not to stop the road roller 100 based on the object detection result received from the object detection unit 405. At this time, it may determine that the road roller 100 is stopped as soon as any object is detected, or may determine that the road roller 100 is stopped when the road roller 100 approaches a predetermined position (distance) relative to the detected object. In addition, it may determine whether or not to stop the road roller 100 depending on the classification of the detected object. For example, it may determine that the road roller 100 is stopped for safety when an obstacle such as a vehicle, person, or traffic cone is detected, and may determine not to stop the road roller 100 when a foreign object such as fallen leaves is detected. When the stop determination unit 407 determines that the road roller 100 is to be stopped, it sends a stop signal to the drive unit 111, causing the road roller 100 to stop traveling.

[0051] Furthermore, when an object is detected by the object detection unit 405, the stop determination unit 407 sends an alarm signal according to the type of the object to the alarm unit 109 to cause the alarm unit 109 to perform an alarm operation. For example, when the object detection unit 405 detects an obstacle such as a person, the stop determination unit 407 sends an alarm signal related to safety to the alarm unit 109 to cause the alarm unit 109 to turn on, for example, a red warning light. When the object detection unit 405 detects a foreign object such as a fallen leaf, the stop determination unit 407 sends an alarm signal related to the work quality to the alarm unit 109 to cause the alarm unit 109 to turn on, for example, a yellow warning light. In this way, it is preferable to cause different alarm operations to be performed according to the type of the detected object. Furthermore, when the alarm unit 109 is a buzzer, it is preferable to generate different buzzer sounds according to the detected object by sending an alarm signal according to the type of the detected object to the alarm unit 109.

[0052] 5 shows a flow chart of a camera control process (image capture control method) executed according to a program by the control unit 108. The control unit 108 repeatedly executes this process while the road roller 100 is traveling to perform work.

[0053] In step S101, the control unit 108 causes the imaging unit 107, which is a front camera, to capture images and acquire imaging data. This imaging data includes imaging data of the traveling direction of the road roller 100 as shown in FIG. 6 and imaging data of the ground (work area) immediately in front of the rolling roller 101.

[0054] Next, in step S102, the control unit 108 causes the development processing unit 403 to perform development processing on the captured data to generate image data, and causes the distortion correction unit 404 to perform distortion correction processing on the image data. Then, the display processing unit 406 generates display image data from the image data after the distortion correction processing, and causes the display unit 110, which is a work monitor, to display the display image. As described above, the display unit 110 displays a display image combining a forward image and a working area image.

[0055] Next, control unit 108 executes the processes of steps S103 to S107 and steps S108 to S112 in parallel. The processes of steps S105 to S107 and steps S110 to S112 correspond to predetermined processes that differ depending on the type of detected object.

[0056] In step S103, the control unit 108 performs object detection processing on the image data of the forward image in the object detection unit 405. At this time, the learning data applied to the object detection processing is limited to obstacles such as people and traffic cones.

[0057] Next, in step S104, the control unit 108 determines whether or not an obstacle was detected in step S103. If an obstacle was not detected, the process ends, and if an obstacle was detected, the process proceeds to step S105.

[0058] In step S105, the control unit 108 calculates the position of the detected obstacle in the display image (forward image) and superimposes an obstacle detection frame surrounding the obstacle on the display image according to the calculation result. This makes it possible to clearly display the presence of the obstacle on the work monitor.

[0059] Next, in step S106, the control unit 108 outputs a safety warning signal from the stop determination unit 407 to the warning unit 109. The warning unit 109 receiving this signal turns on a red warning light and / or generates a buzzer sound to indicate that an obstacle has been detected. This makes it possible to not only notify the operator of the road roller 100, but also surrounding workers, that an obstacle has been detected.

[0060] Next, in step S107, the control unit 108 performs processing to stop the travel (work) of the road roller 100 by sending a stop signal from the stop determination unit 407 to the drive unit 111. At this time, instead of the stopping processing, processing to decelerate the road roller 100 may be performed. This makes it possible to avoid the road roller 100 coming into contact with an obstacle even if the operator of the road roller 100 does not notice the presence of the obstacle.

[0061] On the other hand, in step S108, the control unit 108 performs object detection processing on the image data of the work area image in the object detection unit 405. At this time, the learning data applied to the object detection processing is limited to foreign objects such as fallen leaves and rocks.

[0062] Next, in step S109, control unit 108 determines whether or not a foreign object was detected in step S108. If a foreign object was not detected, this process ends, and if a foreign object was detected, the process proceeds to step S110.

[0063] In step S110, the control unit 108 calculates the position of the detected foreign object in the display image (work area image) and superimposes a foreign object detection frame surrounding the foreign object on the display image according to the calculation result, thereby making it possible to clearly display the presence of the foreign object on the work monitor.

[0064] Next, in step S111, the control unit 108 outputs a warning signal regarding the work quality from the stop determination unit 407 to the warning unit 109. The warning unit 109 receiving this signal turns on a yellow warning light indicating that a foreign object has been detected, or generates a buzzer sound different from that used when an obstacle is detected. This makes it possible to not only notify the operator of the road roller 100, but also surrounding workers, that a foreign object has been detected.

[0065] Next, in step S112, the control unit 108 performs processing to send a stop signal from the stop determination unit 407 to the drive unit 111 to stop the travel (work) of the road roller 100. This makes it possible to avoid compaction of the foreign object by the road roller 100, i.e., a decrease in work quality, even if the operator of the road roller 100 does not notice the presence of the foreign object.

[0066] By carrying out the above camera control processing, it is possible to ensure safety and work quality in front of the road roller 100 using image data obtained through one imaging unit 107 while reducing the burden on the operator.

[0067] In the above embodiment, the resolution is changed according to the angle of view by setting the optical characteristics of the optical system in the imaging unit. In contrast, the resolution may be changed according to the angle of view by changing the pixel density in the imaging element in the imaging unit. Specifically, the pixel density may be increased toward the periphery to make the peripheral side have a higher resolution than the central side. In addition, both the optical characteristics of the optical system and the imaging density of the imaging element may be set so that high resolution imaging can be achieved on the peripheral side of the imaging angle of view rather than the central side.

[0068] As described above, according to this embodiment, both the direction of travel and the work area of ​​the work vehicle can be monitored using image data obtained through a single camera (imaging unit) attached to the front of the work vehicle.

[0069] In the above embodiment, the imaging system is mounted on a work vehicle that travels on the ground, but the work vehicle may be a work vehicle that travels on a track (rail), on the water surface, or in the air. Also, the imaging system similar to the above embodiment may be mounted on various moving bodies such as general automobiles, ships, aircraft, and industrial robots, in addition to work vehicles.

[0070] The above embodiment includes the following configurations.

[0071] (Configuration 1) An imaging unit disposed on the moving body; a control unit that outputs image data acquired by the imaging unit to a display unit, the imaging unit has a configuration capable of imaging with a higher resolution in a peripheral region of an imaging angle of view than in a central region, an image of a first imaging region in a traveling direction of the moving object is captured in a part of the peripheral side region; An imaging system comprising: an imaging unit configured to image a third imaging area in another part of the peripheral side area, the third imaging area being closer to the moving object than the second imaging area imaged in the central side area. (Configuration 2) The imaging unit includes: An imaging element; an optical system that forms an optical image on a light receiving surface of the image sensor; Let f be the focal length of the optical system, y(θ) be the projection characteristic expressing the relationship between the half angle of view θ of the optical system and the image height y on the image plane, and θmax be the maximum half angle of view of the optical system. The optical system includes: 0.20≦2ftan(θmax / 2) / y(θmax) ≦0.95 2. The imaging system according to claim 1, having optical characteristics that satisfy the following conditions: (Configuration 3) The imaging unit includes: An imaging element; an optical system that forms an optical image on a light receiving surface of the image sensor; The imaging system according to configuration 1, wherein the imaging element is configured such that the pixel density of a central region including the center of the light receiving surface of the imaging element is higher than the pixel density of the peripheral region surrounding the central region. (Configuration 4) The control unit is a detection unit that detects an object based on the image data; 4. The imaging system according to any one of configurations 1 to 3, wherein a predetermined process is performed when the object is detected. (Configuration 5) 5. The imaging system according to configuration 4, wherein the control unit outputs, as the predetermined process, image data for allowing a viewer to recognize the object to the display unit. (Configuration 6) having an alarm means for performing an alarm operation, 5. The imaging system according to configuration 4, wherein the control unit causes the warning means to perform the warning operation as the predetermined process. (Configuration 7) 5. The imaging system according to claim 4, wherein the control means performs a process of stopping or decelerating the moving object as the predetermined process. (Configuration 8) The detection unit is a first object that is included in the first imaging area and that may come into contact with the moving object; A second object of a different type from the first object and included in the third imaging area can be detected, The imaging system according to configuration 4, wherein the control unit performs different processes as the predetermined process when the first object is detected and when the second object is detected. (Configuration 9) The moving body is a work vehicle that performs work on a work area while traveling, The third imaging area includes the working area. 13. An imaging system for a moving object according to claim 12, (Configuration 10) The work vehicle is a road roller, 10. The imaging system for a moving body according to configuration 9, wherein the third imaging area includes the ground near a roller portion of the road roller. (Configuration 11) The work vehicle is operated by an operator riding on the work vehicle, 10. The imaging system according to configuration 9, wherein the work area is a blind spot for the operator. (Configuration 12) A moving body comprising the imaging system according to any one of configurations 1 to 11. (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0072] The embodiments described above are merely representative examples, and various modifications and alterations are possible for each embodiment when implementing the present invention. [Explanation of symbols]

[0073] 100 Road Roller 107 Imaging unit 108 Control Unit 110 Display section 401 Optical system 402 Image sensor 405 Object detection unit

Claims

1. An imaging unit disposed on the moving body; a control unit that outputs image data acquired by the imaging unit to a display unit, the imaging unit has a configuration capable of imaging with a higher resolution in a peripheral region of an imaging angle of view than in a central region, an image of a first imaging region in a traveling direction of the moving object is captured in a part of the peripheral side region; An imaging system comprising: an imaging unit configured to image a third imaging area in another part of the peripheral side area, the third imaging area being closer to the moving body than the second imaging area imaged in the central side area.

2. The imaging unit includes: An imaging element; an optical system that forms an optical image on a light receiving surface of the image sensor; Let f be the focal length of the optical system, y(θ) be a projection characteristic that expresses the relationship between the half angle of view θ of the optical system and the image height y on the image plane, and θmax be the maximum half angle of view of the optical system. The optical system includes: 0.20≦2ftan(θmax / 2) / y(θmax)≦0.95 2. The imaging system according to claim 1, wherein the imaging system has optical characteristics that satisfy the following condition:

3. The imaging unit includes: An imaging element; an optical system that forms an optical image on a light receiving surface of the image sensor; 2. The imaging system according to claim 1, wherein the imaging element is configured such that a pixel density in a central region including a center of a light receiving surface of the imaging element is higher than a pixel density in a peripheral region surrounding the central region.

4. The control unit is a detection unit that detects an object based on the image data; 2. The imaging system according to claim 1, wherein a predetermined process is performed when the object is detected.

5. 5. The imaging system according to claim 4, wherein the control unit outputs, as the predetermined process, image data for allowing a viewer to recognize the object to the display unit.

6. having an alarm means for performing an alarm operation, 5. The imaging system according to claim 4, wherein the control unit causes the warning means to perform the warning operation as the predetermined process.

7. 5. The imaging system according to claim 4, wherein the control means performs a process of stopping or decelerating the moving object as the predetermined process.

8. The detection unit is a first object that is included in the first imaging area and that may come into contact with the moving object; A second object of a different type from the first object and included in the third imaging area can be detected, The imaging system according to claim 4 , wherein the control unit performs different processes as the predetermined process when the first object is detected and when the second object is detected.

9. The moving body is a work vehicle that performs work on a work area while traveling, 2. The imaging system for a moving body according to claim 1, wherein the third imaging area includes the working area.

10. The work vehicle is a road roller, 10. The imaging system for a moving body according to claim 9, wherein the third imaging area includes a ground surface in the vicinity of a roller portion of the road roller.

11. The work vehicle is operated by an operator riding on the work vehicle, 10. The imaging system according to claim 9, wherein the work area is a blind spot for the operator.

12. A moving body comprising the imaging system according to any one of claims 1 to 11.

13. An imaging system including an imaging unit disposed on a moving body and a control unit that outputs image data acquired by the imaging unit to a display unit, the imaging unit being configured to capture an image with a higher resolution in a peripheral region of an imaging angle of view than in a central region, an image of a first imaging region in a traveling direction of the moving object is captured in a part of the peripheral side region; An imaging method comprising the steps of: imaging a third imaging area, which is closer to the moving body than a second imaging area imaged in the central area, in another part of the peripheral area.

Images