Work vehicle and image synthesis method for work vehicle

Abstract

To generate a synthetic image from which a user can easily recognize a region behind a vehicle, with a simple algorithm, from two cameras facing diagonally backward.SOLUTION: A work vehicle includes a controller. The controller is configured to generate a trapezoidal image and generate a synthetic image by superimposing the trapezoidal image on a superimposition image obtained from one of first and second images, the trapezoidal image being obtained by deforming a first region of an image obtained from the other of the first and second images into a trapezoidal second region so that a first arrangement direction and a second arrangement direction are substantially orthogonal to each other and a lateral width perpendicular to a vertical direction may be increased as it goes vertically upward, the first arrangement direction in which two imaging targets are arranged, the two imaging targets being spaced from each other in a backward direction and arranged at a substantially equal distance from the first and second cameras in lateral direction, the second arrangement direction in which third and fourth imaging targets are arranged, the third and fourth imaging targets being separated in the lateral direction from a second imaging target which is farther from the work vehicle out of the two imaging targets.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to a work vehicle and an image synthesis method for a work vehicle. [Background technology]

[0002] Patent Document 1 describes a work vehicle that displays an image captured by a camera attached toward the vehicle on a monitor installed in the driver's seat. In this work vehicle, an operator sitting in the driver's seat can see the area behind the vehicle body that is difficult to see from the driver's seat due to the mirror. Patent Document 2 and Patent Document 3 describe work vehicles in which multiple cameras are arranged in the vehicle to widen the angle of view on the rear side of the vehicle body. Patent Document 4 discloses a technology for vertically reducing a specific portion of an image obtained from a camera attached to the left and right rearview mirrors and a camera attached to the center of the rear side of the vehicle body so that the specific portion is the same size as the specific portion obtained from the camera attached to the left and right rearview mirrors. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-079608 A [Patent Document 2] JP 2003-116125 A [Patent Document 3] Patent No. 6551336 [Patent Document 4] Patent No. 4674900 Summary of the Invention [Problem to be solved by the invention]

[0004] In the inventions of Patent Documents 1 and 2, three rearward-viewing cameras are arranged separately, and each is configured to be selectively displayed on the monitor, so it is not possible to check the whole at the same time. In the invention of Patent Document 3, although the images from three cameras are displayed at the same time, the overlapping parts are displayed as they are, making the image difficult for the user to see. The invention of Patent Document 4 has a problem of being costly because it uses three cameras. Note that in Patent Documents 1 and 2, the left and right rear cameras are tilted, so even if the invention of Patent Document 4 is used, it is not possible to generate a composite image of the images from these cameras. [Means for solving the problem]

[0005] A work vehicle according to a first aspect of the present disclosure includes a pair of traveling devices, a vehicle body, a first camera, a second camera, a controller, and a monitor. Each of the pair of traveling devices is configured to move in a forward direction and a backward direction. The vehicle body is provided between the pair of traveling devices in a left-right direction perpendicular to the forward direction and the backward direction. The first camera is provided near the left rear corner of the vehicle body facing the forward direction. The second camera is provided near the right rear corner of the vehicle body facing the forward direction. The controller is configured to generate a composite image by combining a part of a first image captured by the first camera and a part of a second image captured by the second camera. The monitor is configured to display the composite image. The optical axis of the first camera extends in a first direction inclined from the left direction toward the backward direction. The optical axis of the second camera extends in a second direction inclined from the right direction toward the backward direction. In the composite image, the further up in the vertical direction, the more the subject located further back is displayed. The controller is configured to generate a trapezoidal image by deforming a first region of an image obtained from one of the first image and the second image into a trapezoidal second region so that the width in the horizontal direction perpendicular to the vertical direction increases the further upward in the vertical direction, and to superimpose the trapezoidal image on the superimposed image obtained from the other of the first image and the second image, such that a first arrangement direction in which two subjects to be photographed are lined up and spaced apart from each other in the rear direction are substantially perpendicular to a second arrangement direction in which a third subject to be photographed and a fourth subject to be photographed are lined up and spaced apart in the left-right direction from the second subject to be photographed, which is farther from the work vehicle, of the two subjects to be photographed, are lined up.

[0006] A work vehicle according to a second aspect of the present disclosure includes a pair of traveling devices, a vehicle body, a first camera, a second camera, a controller, and a monitor. Each of the pair of traveling devices is configured to move in a forward direction and a backward direction. The vehicle body is provided between the pair of traveling devices in a left-right direction perpendicular to the forward direction and the backward direction. The vehicle body rotatably supports the pair of traveling devices. The first camera is provided near the left rear corner of the vehicle body. The second camera is provided near the right rear corner of the vehicle body. The controller is configured to generate a composite image by combining a part of a first image captured by the first camera and a part of a second image captured by the second camera. The monitor is configured to display the composite image. The optical axis of the first camera extends in a first direction tilted from the left direction toward the backward direction. The optical axis of the second camera extends in a second direction tilted from the right direction toward the backward direction. The controller is configured to generate a trapezoidal image by deforming an image obtained from one of the first image and the second image into a trapezoid shape so as to cover an overlapping portion displayed together in the first image and the second image by using the first camera and the second camera to capture the same subject, and to superimpose the trapezoidal image on an image obtained from the other of the first image and the second image to generate a composite image.

[0007] A composite image generating method for a work vehicle according to a third aspect of the present disclosure includes orienting an optical axis of a first camera, which faces in the forward traveling direction of a travel device of the work vehicle and is provided near the left rear corner of a vehicle body of the work vehicle, in a first direction tilted from the left toward the reverse direction, and orienting an optical axis of a second camera, which faces in the forward traveling direction and is provided near the right rear corner of the vehicle body, in a second direction tilted from the right toward the rear. The composite image generating method includes capturing a first image by the first camera and capturing a second image by the second camera. The composite image generating method includes generating a trapezoidal image by deforming a first region of an image obtained from one of the first image and the second image into a trapezoidal second region such that the width of the composite image in the horizontal direction perpendicular to the vertical direction increases toward the top of the composite image, and generating a composite image by superimposing the trapezoidal image on a superimposed image obtained from the other of the first image and the second image, such that the further up in the vertical direction the composite image is, the wider the width of the composite image in the horizontal direction perpendicular to the vertical direction becomes, and a first arrangement direction of two subjects to be photographed, which are arranged at substantially equal distances from the first camera and the second camera and spaced apart from each other in the rear direction, is substantially perpendicular to a second arrangement direction of a third subject to be photographed and a fourth subject to be photographed, which are arranged at left and right distances from the second subject to be photographed and which are farther from the work vehicle, is arranged.

[0008] A composite image generating method for a work vehicle according to a fourth aspect of the present disclosure includes orienting an optical axis of a first camera provided near a left rear corner of a vehicle body of the work vehicle in a first direction tilted from a left direction toward a rear direction while facing a forward traveling direction of a travel device of the work vehicle, and orienting an optical axis of a second camera provided near a right rear corner of the vehicle body in a second direction tilted from a right direction toward a rear direction while facing a forward traveling direction. The composite image generating method includes capturing a first image by the first camera and capturing a second image by the second camera. The composite image generating method includes generating a trapezoidal image by deforming an image obtained from one of the first image and the second image into a trapezoid shape so as to cover an overlapping portion in which the same photographic subject is displayed in the first image and the second image, and superimposing the trapezoidal image on an image obtained from the other of the first image and the second image to generate a composite image. Effect of the Invention

[0009] According to the technology disclosed in the present application, in order to obtain a wide angle of view behind the vehicle body, a composite image that allows the user to easily view the area behind the vehicle body can be generated using a simple algorithm from two cameras facing diagonally rearward. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a left side view of the work vehicle. [Diagram 2] FIG. 2 is a right side view of the work vehicle. [Diagram 3] FIG. 3 is a top view of the work vehicle. [Figure 4] FIG. 4 is a system configuration diagram of the work vehicle. [Diagram 5] FIG. 5 is a specific example of a composite image. [Figure 6] FIG. 6 is a diagram showing the positional relationship between the work vehicle 1 and the first to eighth subjects to be photographed. [Figure 7] FIG. 7 is a computer graphics model before projective transformation of one of the images. [Figure 8]FIG. 8 is a computer graphics model of one of the images after projective transformation. [Figure 9] FIG. 9 shows one image in the examples of FIGS. [Figure 10] FIG. 10 shows a first rotated image obtained by rotating one of the images in FIG. [Figure 11] FIG. 11 is an example illustrating a projective transformation of a first intermediate image representing a first region of a first rotated image. [Figure 12] FIG. 12 shows an example of how a texture changes when it is subjected to rotation transformation and projective transformation. [Figure 13] FIG. 13 is another example of a projective transformation. [Figure 14] FIG. 14 is another example of a rotation transformation. [Figure 15] FIG. 15 shows the other image in the examples of FIGS. [Figure 16] FIG. 16 shows a second rotated image obtained by rotating the other image of FIG. [Figure 17] FIG. 17 is an example illustrating a projective transformation of a second intermediate image representing a third region of the second rotated image. [Figure 18] FIG. 18 is yet another example of a projective transformation. [Figure 19] FIG. 19 is yet another example of a rotation transformation. [Figure 20] FIG. 20 is an example of a flowchart of the composite image generating method according to the first embodiment. [Figure 21] FIG. 21 is an example of a flowchart of the composite image generating method according to the first embodiment. [Figure 22] FIG. 22 is another example of a flowchart of the composite image generating method according to the first embodiment. [Diagram 23] FIG. 23 is another example of a flowchart of the synthetic image generating method according to the first embodiment. [Figure 24] FIG. 24 shows a case where a non-display area is included in the first area of ​​the first rotated image. [Diagram 25]FIG. 25 shows a case where a non-display area is included in the third area of ​​the second rotated image. [Figure 26] FIG. 26 shows a case where a non-display area is included in the second area of ​​the first rotated trapezoidally transformed image. [Figure 27] FIG. 27 shows a case where a non-display area is included in the fourth area of ​​the second rotated trapezoidally transformed image. [Figure 28] FIG. 28 is an example of a flowchart of a synthetic image generating method according to the second embodiment. [Figure 29] FIG. 29 is an example of a flowchart of a synthetic image generating method according to the second embodiment. [Diagram 30] FIG. 30 is another example of a flowchart of the synthetic image generating method according to the second embodiment. [Diagram 31] FIG. 31 is another example of a flowchart of the synthetic image generating method according to the second embodiment.

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the drawings showing embodiments thereof. In the drawings, the same reference numerals indicate corresponding or substantially identical components. First Embodiment <Overall composition>

[0012] 1 to 3, a work vehicle 1, for example, a compact track loader, includes a vehicle body 2, a pair of traveling devices 3, a working device 4, and a cabin 5. The vehicle body 2 supports the pair of traveling devices 3, the working device 4, and the cabin 5. In the illustrated embodiment, the pair of traveling devices 3 are crawler-type traveling devices. Therefore, the pair of traveling devices 3 each include a driving wheel 31, driven wheels 32 and 33, and a roller 34. However, the pair of traveling devices 3 is not limited to crawler-type traveling devices. The pair of traveling devices 3 may be, for example, front-wheel / rear-wheel traveling devices, or traveling devices having front wheels and rear crawlers. The working device 4 includes a work equipment (bucket) 41 at a distal end of the working device 4. A proximal end of the working device 4 is attached to the rear of the vehicle body 2. The working device 4 includes a pair of arm mechanisms 42 for rotatably supporting the bucket 41 via a bucket pivot shaft 43. Each of the pair of arm mechanisms 42 includes a link 44 and an arm 45 .

[0013] The link 44 is rotatably supported by the vehicle body 2 via a fulcrum shaft 46. The arm 45 is rotatable relative to the link 44 around a joint shaft 47. The working device 4 further includes a plurality of arm cylinders 48 and at least one equipment cylinder 49. Each of the plurality of arm cylinders 48 is rotatably connected to the vehicle body 2 and the arm 45, and moves the link 44 and the arm 45 to raise and lower the bucket 41. The at least one equipment cylinder 49 is configured to tilt the bucket 41. The cabin 5 is attached to the front of the vehicle body 2. The work vehicle 1 includes a window 51 in front of the cabin 5, and includes a driver's seat 52 and a monitor 53 in the cabin 5. That is, the driver's seat 52 and the monitor 53 are provided on the vehicle body 2. The monitor 53 includes an input interface 53i for switching the screen to be displayed, etc. For example, when the monitor 53 is a touch panel, the input interface 53i is a touch sensor. Alternatively, the input interface 53i may be an interface such as a button, a key, or a dial attached to the monitor 53. The window 51 is a front window of the cabin 5. The cabin 5 is defined by a cabin frame 7. As shown in FIG. 2, the cabin frame 7 is rotatable about rotational shafts RSL and RSR on the vehicle body 2. In FIG. 1 and FIG. 2, a common rotational axis A defined by the rotation shafts RSL and RSR is a common rotational axis A. XC is illustrated.

[0014] In the embodiment of the present application, FB (Forward direction D F / backward D B ) means the front-rear direction (front direction / rear direction) as seen from the operator seated in the driver's seat 52 of the cabin 5. L , right direction D R , left / right direction D W These mean the left, right, and left / right directions, respectively, as seen from the operator. L , right direction DR , left / right direction D W is the forward direction D F and backward D B and perpendicular to the upward direction D U , downward D D , height direction D H means the upward, downward, and height directions from the operator's point of view. U , downward D D , height direction D H The forward direction D F and backward D B and left direction D L and right direction D R The front-rear / left-right (lateral) / up-down (height) directions of the work vehicle 1 respectively correspond to the front-rear / left-right (lateral) / up-down (height) directions as seen by the operator.

[0015] Fig. 1 shows the left side of the work vehicle 1. Fig. 2 shows the right side of the work vehicle 1. Each of the pair of traveling devices 3 is rotatable in the forward direction D F and backward D B The vehicle body 2 is configured to move in the left-right direction D W The pair of traveling devices 3 are provided between the pair of traveling devices 3 in the forward direction D F A first traveling device 3L is disposed on the left side of the vehicle body 2 facing in the forward direction D F 1 and 2, the vehicle body 2 is arranged in the left-right direction D. W The vehicle body 2 has a pair of side surfaces 2S facing each other. The pair of side surfaces 2S includes a first side surface 2L which is the left side surface of the vehicle body 2, and a second side surface 2R which is the right side surface of the vehicle body 2. The vehicle body 2 supports a pair of traveling devices 3 (first traveling device 3L, second traveling device 3R) on the pair of side surfaces 2S (first side surface 2L, second side surface 2R), respectively. Specifically, the vehicle body 2 supports the first traveling device 3L on the first side surface 2L, and supports the second traveling device 3R on the second side surface 2R.

[0016] As shown in Fig. 3, the vehicle body 2 is generally symmetrical with respect to the vehicle body center plane M. In Fig. 3, a driver's seat 52 and a monitor 53 inside the cabin 5 are illustrated by dotted lines. With reference to Fig. 3, each of the pair of arm mechanisms 42 is oriented in the left-right direction D. W The pair of arm mechanisms 42 are provided between the pair of traveling devices 3 and the driver's seat 52. W a first arm mechanism 42L located between the first traveling device 3L and the driver's seat 52 in the left-right direction D W and a second arm mechanism 42R located between the second traveling device 3R and the driver's seat 52 in the vehicle body 2. As shown in Figs. 1 to 3, the link 44 included in the first arm mechanism 42L is referred to as a first link 44L, and the arm 45 included in the first arm mechanism 42L is referred to as a first arm 45L. The link 44 included in the second arm mechanism 42R is referred to as a second link 44R, and the arm 45 included in the second arm mechanism 42R is referred to as a second arm 45R. The fulcrum shaft 46 about which the first arm mechanism 42L rotates relative to the vehicle body 2 is referred to as a first fulcrum shaft 46L, and the fulcrum shaft 46 about which the second arm mechanism 42R rotates relative to the vehicle body 2 is referred to as a second fulcrum shaft 46R. The joint shaft 47 around which the first arm 45L rotates relative to the first link 44L is referred to as a first joint shaft 47L, and the joint shaft 47 around which the second arm 45R rotates relative to the second link 44R is referred to as a second joint shaft 47R. With reference to FIGS. 1 and 2, the first fulcrum shaft 46L is located rearward D from the rear end 3LE of the first traveling device 3L. B The second fulcrum shaft 46R is provided at a position D rearward of the rear end 3RE of the second traveling device 3R. B It will be established.

[0017] 1 and 2, the work vehicle 1 further includes an engine 6 provided at the rear of the vehicle body 2. The engine 6 is configured to provide driving force to the traveling device 3 and the working device 4. The engine 6 rotates in the left-right direction D of the work vehicle 1. WIt is provided between a pair of arm mechanisms 42. The work vehicle 1 further includes a cover 8 for covering the engine 6. The work vehicle 1 further includes a bonnet cover 9 provided at the rear end of the vehicle body 2. The bonnet cover 9 is openable and closable, and maintenance personnel can perform maintenance work on the engine 6 and the like.

[0018] Referring to FIGS. 1 to 3, the work vehicle 1 further includes a first camera 30L and a second camera 30R for photographing the external environment of the work vehicle 1. The first camera 30L and the second camera 30R are preferably fisheye cameras or ultra-wide-angle cameras. The first camera 30L is provided in the vicinity of the left rear corner of the vehicle body 2. The position of the first camera 30L in the present embodiment is defined by the lens focal position F1 of the first camera 30L. That is, the lens focal position F1 is located in the vicinity of the left rear corner of the vehicle body 2 toward the front direction D F Specifically, the first camera 30L is provided between the driver's seat 52 and the first traveling device 3L in the left-right direction D W and is provided rearward D B of the rear end 3LE of the first traveling device 3L. That is, the lens focal position F1 is between the left end 52LE of the driver's seat 52 and the left end 3LLE of the first traveling device 3L in the left-right direction D W and is located rearward D B of the rear end 3LE of the first traveling device 3L. More specifically, the first camera 30L is provided between the driver's seat 52 and the first arm mechanism 42L in the left-right direction D W and is provided rearward D B of the first fulcrum shaft 46L. That is, the lens focal position F1 is between the left end 52LE of the driver's seat 52 and the right end 42LRE of the first arm mechanism 42L in the left-right direction D W and is located rearward D B of the first fulcrum shaft 46L. When viewed in the height direction D H the first camera 30L and the second camera 30R overlap the bonnet cover 9.

[0019] The second camera 30R is provided near the right rear corner of the vehicle body 2. The position of the second camera 30R in the present embodiment is defined by the lens focal point position F2 of the second camera 30R. That is, the lens focal point position F2 is located near the right rear corner of the vehicle body 2 toward the forward direction D F Specifically, the second camera 30R is provided between the driver's seat 52 and the second traveling device 3R in the left-right direction D W and is provided rearward D B of the rear end 3RE of the second traveling device 3R. That is, the lens focal point position F2 is between the right end 52RE of the driver's seat 52 and the right end 3RRE of the second traveling device 3R in the left-right direction D W and is located rearward D B of the rear end 3RE of the second traveling device 3R. More specifically, the second camera 30R is provided between the driver's seat 52 and the second arm mechanism 42R in the left-right direction D W and is provided rearward D B of the second fulcrum shaft 46R. That is, the lens focal point position F2 is between the right end 52RE of the driver's seat 52 and the left end 42RLE of the second arm mechanism 42R in the left-right direction D W and is located rearward D B of the second fulcrum shaft 46R.

[0020] The optical axis Ax1 of the first camera 30L extends in a first direction D1 inclined from the left direction D L toward the rear direction D B The optical axis Ax2 of the second camera 30R extends in a second direction D2 inclined from the right direction D R toward the rear direction D B The angle θ formed by the optical axis Ax1 of the first camera 30L and the rear direction D B and the angle θ formed by the optical axis Ax2 of the second camera 30R and the rear direction D B are preferably 15 degrees or more and 60 degrees or less. The viewing angle α of the first camera 30L and the viewing angle β of the second camera 30R are preferably 120 degrees or more. For this reason, a part of the area that the first camera 30L can photograph and a part of the area that the second camera 30R can photograph overlap. FIG. 3 shows the overlapping area as area S.

[0021] FIG. 4 is a system configuration diagram of the work vehicle 1. Referring to FIG. 4, the work vehicle 1 includes a controller 10 that processes a first image IMGL captured by the first camera 30L and a second image IMGR captured by the second camera 30R. The controller 10 includes a hardware processor 11, a memory 12, a bus 18, and an external interface 19. The memory 12 includes a non-volatile area configured to store an image processing program 15 that inputs the first image IMGL and the second image IMGR and outputs a composite image CI, and a volatile area configured to temporarily load work data such as the program. The hardware processor 11 is configured to execute the image processing program 15. The external interface 19 inputs signals from the first camera 30L, the second camera 30R, and the input interface 53i, and outputs a signal for displaying the composite image CI on the monitor 53. The input interface 53i includes, for example, HDMI (registered trademark), IEEE1394, USB, Ethernet, RS-232C, and the like. Control signals and data between the hardware processor 11 and various devices are sent via a bus 18. The various devices include, for example, a memory 12, a first camera 30L, a second camera 30R, an input interface 53i, and a monitor 53. The controller 10 may further include an image processor for quickly performing part of the image processing of the first image IMGL and the second image IMGR, and a graphics board for quickly drawing the composite image CI. The controller 10 may be referred to as a control circuitry.

[0022] The controller 10 (hardware processor 11 executing the image processing program 15) is configured to generate a composite image CI by combining a part of the first image IMGL and a part of the second image IMGR. The monitor 53 is configured to generate the composite image CI. Specifically, the controller 10 (hardware processor 11 executing the image processing program 15) is configured to generate a composite image CI as shown in Fig. 5. The first to eighth photographic subjects P1 to P8 appear in the composite image CI.

[0023] 6 is a diagram showing the positional relationship between the work vehicle 1 and the first to eighth photographing targets P1 to P8. The first photographing target P1 is the midpoint between the center of the part of the triangular cone of the fifth photographing target P5 that contacts the ground and the center of the part of the triangular cone of the sixth photographing target P6 that contacts the ground. FB and the distance between the sixth photographing object P6 and the rear end 9B of the work vehicle 1 in the front-rear direction D FB The distance between the third subject P3 and the rear end 9B of the work vehicle 1 in the front-rear direction D is equal to the distance between the third subject P3 and the rear end 9B of the work vehicle 1 in the front-rear direction D. FB and the distance between the fourth photographing object P4 and the rear end 9B of the work vehicle 1 in the front-rear direction D FB The fifth subject P5 and the sixth subject P6 are closer to the rear end 9B of the work vehicle 1 than the third subject P3 and the fourth subject P4.

[0024] Two subjects P1 and P2 are in the left-right direction D W The first camera 30L and the second camera 30R are disposed at substantially equal distances from each other in the rear direction D. B The third to sixth subjects P3 to P6 are arranged on the ground so as to be spaced apart from each other in the forward and backward direction D. The triangular cone of the seventh subject P7 and the triangular cone of the eighth subject P8 are arranged closer to the rear end 9B of the work vehicle 1 than the fifth subject P5 and the sixth subject P6. FB The position of is in the front-rear direction D of the rear end 9B of the work vehicle 1. FB is equal to the position of

[0025] 6 shows the angle of view α of the first camera 30L and the angle of view β of the second camera 30R. The first to sixth objects P1 to P6 are photographed by both the first camera 30L and the second camera 30R. The seventh object P7 is photographed only by the first camera 30L. The eighth object P8 is photographed only by the second camera 30R. <Synthetic image generation algorithm>

[0026] As shown in FIG. 5, the controller 10 (hardware processor 11 executing image processing program 15) is configured to generate a trapezoidal image TZPG1 by deforming an image obtained from one image IMG1 of the first image IMGL and the second image IMGR into a trapezoid shape so as to cover the overlapping portion OVR displayed together in the first image IMGL and the second image IMGR by the first camera 30L and the second camera 30R capturing the same subject (e.g., the fourth subject P4), and to superimpose the trapezoidal image TZI1 on an image obtained from the other image IMG2 of the first image IMGL and the second image IMGR to generate a rectangular composite image CI.

[0027] 7 and 8 are computer graphics models for generating a composite image CI. FIG. 7 shows a model before projective transformation of one image IMG1, and FIG. 8 shows a model after projective transformation of one image IMG1. In these computer graphics models, the first image IMGL is the texture of a planar first polygon PLG1, and the second image IMGR is the texture of a planar second polygon PLG2. Of the first polygon PLG1 and the second polygon PLG2, the polygon corresponding to one image IMG1 is placed on the side closer to the virtual camera VC. Although FIG. 5 and FIG. 7 show an example in which one image IMG1 is the first image IMGL and the other image IMG2 is the second image IMGR, one image IMG1 may be the second image IMGR and the other image IMG2 may be the first image IMGL.

[0028] The composite image CI is an image in which an object in a virtual space determined by the first polygon PLG1 and the second polygon PLG2 is projected onto a projection plane PP between the virtual camera VC and the first polygon PLG1 and the second polygon PLG2. The projection plane PP, the first polygon PLG1, and the second polygon PLG2 are displayed in the virtual space parallel to each other. Here, a two-dimensional coordinate system in the virtual space based on the vertical and horizontal directions of the projection plane PP is referred to as a reference coordinate system D. V -D SEven if such a computer graphics model is not used and the composite image CI is simply output, the vertical direction D V and the horizontal direction D S The two-dimensional coordinate system associated with V -D S It may also be called.

[0029] In generating the composite image CI as shown in Fig. 8, the first polygon PLG1 is deformed into a first deformed polygon TZPLG1 to match the shape of the trapezoid image TZI1. The second polygon PLG2 is also deformed into a second deformed polygon TZPLG2 in the same manner as the first polygon PLG1. The other image IMG2 is also deformed into a trapezoid shape in the same manner as the one image IMG1, as will be described in detail later. The image obtained by deforming the other image IMG2 is called an added trapezoid image TZI2.

[0030] More specifically, referring to FIGS. 5 to 8, the controller 10 (the hardware processor 11 executing the image processing program 15) performs the vertical direction D V The higher you go, the more backward the D B The controller 10 (the hardware processor 11 for executing the image processing program 15) generates a composite image CI in which the two subjects P1 to P2 are arranged in a first arrangement direction (the vertical direction D in FIG. 5). V ) and from the second photographing target P2, which is farther from the work vehicle 1 among the two photographing targets P1 to P2, in the left and right direction D W The third and fourth objects P3 and P4 are arranged in a second arrangement direction (the horizontal direction D in FIG. 5). S ) is substantially perpendicular to the vertical direction D V The higher you go, the more vertical D V The horizontal direction D perpendicular to SThe controller 10 is configured to generate a trapezoidal image TZI1 by transforming a rectangular first region RG1 (see FIG. 7, etc.) of an image obtained from one image IMG1 of the first image IMGL and the second image IMGR into a trapezoidal second region RG2 (see FIG. 8, etc.) so that the width of the trapezoidal image TZI1 is increased. "The first arrangement direction and the second arrangement direction are substantially perpendicular to each other" means that the composite image CI is located in the center so that a general operator who sees the composite image CI recognizes that the first arrangement direction and the second arrangement direction are perpendicular to each other. The controller 10 (hardware processor 11 that executes the image processing program 15) is configured to generate the composite image CI by superimposing the trapezoidal image TZI1 on a superimposed image (additional trapezoidal image TZI2) obtained from the other image IMG2 of the first image IMGL and the second image IMGR. A method for generating such a composite image CI will be described in detail below.

[0031] Fig. 9 shows one image IMG1 (first image IMGL) in the examples of Fig. 5, Fig. 7, and Fig. 8. Referring to Fig. 9, the controller 10 (the hardware processor 11 executing the image processing program 15) detects whether the first photographed object P1 is in the lateral direction D of the composite image CI. S The first reference point O is determined to be substantially located at the center of the first region RG1. The position of the first reference point O is not changed when the first region RG1 is transformed into the second region RG2. A The trapezoidal image TZI1 is generated based on the position in an image (such as a first rotated image ROI1 and a first rotated trapezoidal image TZRO1, which will be described later) obtained from one of the images IMG1. S "Substantially located at the center of the composite image CI" means that a general operator who sees the composite image CI can see that the first photographing target P1 is located at the center of the composite image CI in the lateral direction D. S It means that it is so central that it is recognized as being in the center of the

[0032] The controller 10 (the hardware processor 11 that executes the image processing program 15) BThe first reference point O in an image (e.g., a first rotated image ROI1, a first rotated trapezoidally transformed image TZRO1, etc.) obtained from one image IMG1 of the first image IMGL and the second image IMGR, which is determined so that a portion closer to the work vehicle 1 than the first photographed object P1 (e.g., the seventh photographed object P7) is included in the trapezoidal image TZI1. A The trapezoidal image TZI1 is generated based on the

[0033] The controller 10 (the hardware processor 11 executing the image processing program 15) detects the vertical direction D V The position of the fourth object P4 in the composite image CI is in the vertical direction D V The image processing apparatus is configured to generate a trapezoidal image TZI1 by transforming a rectangular first region RG1 (first intermediate image IIMG1) of a first rotated image ROI1 obtained by rotating one image IMG1 of the first image IMGL and the second image IMGR into a trapezoidal second region RG2. FIG. 10 shows the first rotated image ROI1. FIG. 11 is an example showing the projective transformation of the first intermediate image IIMG1. Note that FIG. 9 illustrates a portion of one image IMG1 that corresponds to the first region RG1.

[0034] Here, the first reference point O, whose position does not change when the first region RG1 is transformed into the second region RG2, A The position of the first reference point O refers to a point at which the position of the corresponding pixel does not change before and after the projective transformation in Fig. 11, and when one image IMG1 is the first image IMGL, it refers to one of the bottom or left sides of the rectangular periphery of the first intermediate image IIMG1 and the trapezoid periphery of the trapezoid image TZI1 shown in Fig. 11. More preferably, the first reference point O A The position of the first reference point O is the lower left corner of these perimeters. A The position of the first reference point O is one of the lower or right sides of the rectangle and the trapezoid. A The location of is the lower right corner of these perimeters.

[0035] In Fig. 10, to express the rotation direction, the vertical and horizontal orientation of the image after rotation transformation is D' V -D' S In FIG. 10, the vertical direction D in the composite image CI of the third photographed object P3 is V The position of the fourth object P4 in the composite image CI is in the vertical direction D V The rotation angle θ1 substantially coincides with the position in the first rotated image ROI1. This θ1 can be calculated from the inclination of the straight line P3P4 in the first image IMG1 by a well-known method, and therefore a detailed explanation is omitted. Once θ1 is determined, it is stored in the memory 12, and the hardware processor 11 executing the image processing program 15 may read θ1 from the memory 12 and generate the first rotated image ROI1 from the first image IMG1. Note that in FIG. 10, the rotation center O R1 is the origin of one image IMG1. B The case where the center of rotation is O R1 is the origin of one image, IMG1. B The rotation in this case is realized by a combination of a rotation transformation and a translation.

[0036] Next, the projective transformation in Fig. 11 will be described. In Fig. 11, the position of a pixel in the first intermediate image IIMG1 is determined based on the first reference point O A Reference coordinate system D with origin (0,0) V -D S In this case, the coordinates of the first photographed object P1 in the above figure before the projective transformation are (u1, v1), the coordinates of the third photographed object P3 are (u3, v2), and the coordinates of the fourth photographed object P4 are (u4, v2). S The length of the first intermediate image IIMG1 (first region RG1) is a, and the vertical direction D V The length of the composite image CI is defined as b. V In the rotation transformation, the length of the third object P3 in the vertical direction D V and the vertical direction D of the fourth shooting target P4 V Since the positions at P3 and P4 are substantially the same, the v coordinate of P3 and the v coordinate of P4 are represented by the same v2.

[0037] At this time, if the upper side of the image is projected to be t times larger as shown in the lower diagram of Fig. 11, the coordinates of P1 are changed from (u1,v1) to (u1+(t-1)*u1*v1 / b,v1), the coordinates of P3 are changed from (u3,v2) to (u3+(t-1)*u3*v2 / b,v2), and the coordinates of P4 are changed from (u4,v2) to (u4+(t-1)*u4*v2 / b,v2). At this time, the first arrangement direction AD1 in which the two subjects P1-P2 are arranged and the second arrangement direction AD2 in which the third subject P3 and the fourth subject P4 are arranged in Fig. 11 are substantially perpendicular to each other, i.e., the horizontal direction D of the two subjects P1-P2 are substantially perpendicular to each other. S Since the positions in substantially coincide with each other, formula 1 holds.

number

number

[0038] However, as explained above, the first reference point O after the rotation transformation A However, (1) the first photographed object P1 is in the horizontal direction D of the composite image CI. S (2) the first subject P1 is located in the rear direction D B The first reference point O is set so as to satisfy two conditions that a part close to the work vehicle 1 (in the example of FIG. 11, the seventh photographing target P7) is included in the trapezoid image TZI1. A The position of the first photographed object P1 must be determined. However, whether these conditions are satisfied or not depends on whether the first photographed object P1 is located in the lateral direction D of the composite image CI. S Whether t is actually located in the center of t can only be determined by performing the projective transformation in Figure 11. Therefore, (Equation 2) cannot be solved algebraically. Therefore, a search algorithm can be used to find the solution for t as follows:

[0039] First, when the first rotated image ROI1 is obtained, the first reference point O in the first rotated image ROI1 is determined so as to satisfy the condition (2). A The vertical direction of D V Position v A By determining v1 and v2, v1 and v2 can be determined. Also, from the condition (1), the horizontal direction D of the first shooting object P1 after the projective transformation is S The position of is the horizontal direction D of the composite image CI. S Since the length is half of w / 2,

number

number

number

number

[0040] Thus, the first reference point O AThe optimal position of is obtained by sequential search after rotation transformation. The first region RG1 shown in FIG. 9, FIG. 10, and FIG. 11 is the first reference point O determined in this way. A The position of the optimal first reference point O A Once this is determined, it can be said that the controller 10 (hardware processor 11 executing the image processing program 15) generates a trapezoidal image TZI1 by transforming the first region RG1 defined thereby (see Figures 9, 10, and 11) into a trapezoidal second region RG2 (see Figure 11).

[0041] In computer graphics systems, when a polygon is transformed into a trapezoid, one of the images IMG1 that is the source of the texture may be projected. For example, when the first region RG1 shown in FIG. 12 is pasted onto the first polygon PLG1 and the first polygon PLG1 is enlarged to the first transformed polygon TZPLG1, the horizontal direction D of one of the images IMG1 may change. S If an image cut out from the second region RG2 of the image (first rotated trapezoidal transformed image TZRO1) that has been projected and then rotated is pasted as a texture onto the first deformed polygon TZPLG1, the enlargement direction may shift relatively. In this case, the following processing may be performed to obtain the desired trapezoidal image TZI1.

[0042] By utilizing the fact that the angle between the straight lines P1P2 and P3P4 does not change during rotational transformation, the controller 10 (hardware processor 11 executing the image processing program 15) performs projective transformation on one of the first image IMGL and the second image IMGR, image IMG1, to become a trapezoid so that the first arrangement direction AD1 and the second arrangement direction AD2 are substantially perpendicular to each other, and then converts the projectively transformed image (one of the trapezoidally corrected images TZO1) into a trapezoidal shape in the vertical direction D in the composite image CI of the third photographed object P3. V The position of the fourth object P4 in the composite image CI is in the vertical direction D VThe present invention is configured to generate a trapezoidal image TZI1 obtained by extracting the second region RG2 from the first rotated trapezoidally transformed image TZRO1 rotated so as to substantially coincide with the position in the first region RG2. By performing the conversion in this manner, a trapezoidal image TZI1 as shown in the lower diagram of FIG. 11 can be obtained. This method will be described below. FIG. 13 is a diagram for explaining the projective transformation in this case, and FIG. 14 is a diagram for explaining the rotational transformation in this case.

[0043] In FIG. 13, the pixel position of one image IMG1 before projective transformation is set to the bottom left corner as the origin O. B The coordinates of the third object P3 before the projective transformation are (u3, v3), the coordinates of the fourth object P4 before the projective transformation are (u4, v4), the coordinates of the fifth object P5 before the projective transformation are (u5, v5), and the coordinates of the sixth object P6 before the projective transformation are (u6, v6). S The length of the image IMG1 is c, and the vertical direction of the image IMG1 is D. V Let the length of be d.

[0044] In this case, if the top side of the image is projected to be t times larger (t: stretching coefficient) than the original image to form a trapezoid, as shown in Figure 13 (for ease of explanation, this type of transformation may be referred to as trapezoid correction), the coordinates of P3 will change from (u3,v3) to (u3+(t-1)*u3*v3 / d,v3), the coordinates of P4 will change from (u4,v4) to (u4+(t-1)*u4*v4 / d,v4), the coordinates of P5 will change from (u5,v5) to (u5+(t-1)*u5*v5 / d,v5), and the coordinates of P6 will change from (u6,v6) to (u6+(t-1)*u6*v6 / d,v6). In this case, let k1=(u5+u6―u3―u4) / 2, k2=(u5v5+u6v6―u3v3―u4v4) / 2, k3=(v5+v6―v3―v4) / 2, k4=u4―u3, k5=u4v4―u3v3, k6=v4―v3.

number

number

number

number

[0045] When the stretching coefficient t is calculated in this manner, the vertical direction D of the third subject P3 is V and the vertical direction D of the fourth shooting target P4 V Rotate one of the keystone correction images TZO1 so that the positions at the center of the image are substantially the same as those at the center of the image O. R2 The third object P3 is rotated around the vertical direction D to obtain the first rotated trapezoidal transformed image TZRO1. V and the vertical direction D of the fourth shooting target P4 V The rotation angle when one of the trapezoid corrected images TZO1 is rotated so that the positions in and substantially match is defined as θ2. This θ2 can be obtained by a well-known method from the inclination of the straight line P3P4 in one of the trapezoid corrected images TZO1, and therefore a detailed explanation is omitted. θ2 determined in this manner is stored in the memory 12, and the hardware processor 11 executing the image processing program 15 may read θ2 from the memory 12 and generate a first rotated trapezoidally transformed image TZRO1 from one of the trapezoid corrected images TZO1. The rotation center O in this rotation transformation R2 is the origin O of one of the keystone correction images TZO1 as shown in Figs.B The present invention is not limited to this, and may be applied to other points.

[0046] In this case, (1) the first photographed object P1 is in the lateral direction D of the composite image CI. S (2) the first subject P1 is located in the rear direction D B The first reference point O in the first rotated trapezoidally transformed image TZRO1 is determined so as to satisfy two conditions: that the part close to the work vehicle 1 (the seventh photographed object P7) is included in the trapezoidal image TZI1. A At this time, the position of the first target P1 displayed in the first rotated trapezoidal transformed image TZRO1 in the horizontal direction D S From the position of the synthetic image CI, the horizontal direction D S The position shifted by half the length w / 2 is the first reference point O A Horizontal direction D S If the first reference point O is located at this position, condition (1) is satisfied. The first reference point O is located at this position so that either the seventh object P7 or the eighth object P8 displayed in the first rotated trapezoidally transformed image TZRO1 is included in the second region RG2. A The vertical direction of D V If the position of is determined, condition (2) is met.

[0047] The lateral direction D of the first region RG1 S The length of the first region RG1 is a, and the length of the first region RG1 is D V The length of the image IMG1 is b, and the horizontal direction D S The length of the image IMG1 is c, and the vertical direction of the image IMG1 is D. V If the length of the first reference point O in the first rotated trapezoidal transformed image TZRO1 is d, the length of the upper side of the second region RG2 is ab / d times the solution t of (Equation 4), and the shape of the second region RG2 can be determined. A Position (u A , v A When the first reference point O in the first rotated trapezoidally transformed image TZRO1 is determined, the position and shape of the second region RG2 is specified. AThe position and the size of the second region RG2 may be stored in memory 12, and the hardware processor 11 executing the image processing program 15 may read these values ​​from memory 12 to generate a trapezoidal image TZI1 from the first rotated trapezoidally transformed image TZRO1.

[0048] The first reference point O shown above A The optimal position of the first reference point O shown in FIG. 9 will be obtained after keystone correction and rotation transformation. A The position of the first reference point O in the first rotated trapezoidally transformed image TZRO1 determined as described above. A The position of the first reference point O in the first rotated trapezoidally transformed image TZRO1 is obtained by performing an inverse transformation of the rotation transformation and an inverse transformation of the trapezoidal correction. A A first reference point O in one image IMG1 corresponding to the position of A The shape of the first region RG1 shown in FIG. 9 can be considered to be equal to the shape obtained by performing the inverse conversion of the keystone correction described above on the shape of the second region RG2 in FIG. 14. A When the first region RG1 is determined, the controller 10 (the hardware processor 11 executing the image processing program 15) generates a trapezoidal image TZI1 by transforming the first region RG1 (see FIG. 9) defined thereby into a trapezoidal second region RG2 (see FIG. 14). V The length b of the composite image CI is the vertical direction D V The length of the first region RG1 and the second region RG2 in the lateral direction D S The length of the second region RG2 in the horizontal direction D S The length of the composite image CI is the horizontal direction D Smay be arbitrarily set to exceed half the length w / 2 of the first rotated trapezoidally transformed image TZRO1. In this case, the first region RG1 in the one image IMG1 corresponding to the second region RG2 in the first rotated trapezoidally transformed image TZRO1 is not necessarily rectangular. In this case, it can be said that the controller 10 (hardware processor 11 executing the image processing program 15) generates a trapezoidal image TZI1 by transforming the first region RG1 in the one image IMG1 corresponding to the second region RG2 in the first rotated trapezoidally transformed image TZRO1 into the trapezoidal second region RG2 shown in the first rotated trapezoidally transformed image TZRO1.

[0049] As described above, after forming the trapezoidal image TZI1, the controller 10 (the hardware processor 11 executing the image processing program 15) is configured to generate an additional trapezoidal image TZI2 (see FIG. 8) from the other image IMG2 (in the above example, the second image IMGR) using an algorithm similar to that for the trapezoidal image TZI1. Specifically, the controller 10 (the hardware processor 11 executing the image processing program 15) is configured to generate an additional trapezoidal image TZI2 (see FIG. 8) from the other image IMG2 (in the above example, the second image IMGR) using an algorithm similar to that for the trapezoidal image TZI1. V The higher you go, the more horizontal D S The method is configured to generate an additional trapezoidal image TZI2 as a superimposed image by transforming the rectangular third region RG3 of the other image IMG2 of the first image IMGL and the second image IMGR into a trapezoidal fourth region RG4 so that the width of the image becomes wider. The method of generating the additional trapezoidal image TZI2 will be described below, focusing on the differences from the method of generating the trapezoidal image TZI1. Note that the contents described in the method of generating the trapezoidal image TZI1 are similarly applied to the parts that are omitted in the following description.

[0050] FIG. 15 shows the other image IMG2 (the second image IMGR in the example of FIG. 15) in the examples of FIGS. 5, 7, and 8. As in the examples of FIGS. 9 and 10, the controller 10 (the hardware processor 11 executing the image processing program 15) calculates the vertical direction D in the composite image CI of the third photographing object P3 from the image as shown in FIG. V The position of the fourth object P4 in the composite image CI is in the vertical direction D VThe other image IMG2 is rotated around the center O so that the position of the image IMG2 substantially coincides with the position of the image IMG2 at the center O. R3 and generating an additional trapezoidal image TZI2 by projectively transforming an image (second intermediate image IIMG2) corresponding to the third region RG3 of the second rotated image ROI2 obtained by rotating the image around the trapezoidal shape. FIG. 16 shows the second rotated image ROI2. FIG. 17 is an example showing the projective transformation of the second intermediate image IIMG2. Note that FIG. 15 illustrates a portion of the other image IMG2 that corresponds to the third region RG3.

[0051] In the following description of the embodiment, a point whose position does not change when the third region RG3 is transformed into the fourth region RG4 is referred to as a second reference point O. C The second reference point O C The position of the second reference point O indicates, for example, a point at which the position of the corresponding pixel does not change before and after the projective transformation in Fig. 17, and indicates one point on the bottom side or the right side of the outer periphery of the rectangle of the second intermediate image IIMG2 or the outer periphery of the trapezoid of the additional trapezoid image TZI2 shown in Fig. 17 when the other image IMG2 is the second image IMGR. C The position of the second reference point O is the lower right corner of these perimeters. C The position of the second reference point O is one of the lower side or the left side of the rectangle or trapezoid. C The location of is the lower left corner of these perimeters.

[0052] In Fig. 16, to express the rotation direction, the vertical and horizontal orientation of the image after rotation transformation is D' V -D' S In FIG. 16, the vertical direction D in the composite image CI of the third photographed object P3 is V The position of the fourth object P4 in the composite image CI is in the vertical direction D VThe rotation angle that substantially coincides with the position in the other image IMG2 is represented as θ3. This θ3 can be found by a well-known method from the gradient of the straight line P3P4 in the other image IMG2, and so a detailed explanation is omitted. Once θ3 is determined, it is stored in the memory 12, and the hardware processor 11 executing the image processing program 15 may read θ3 from the memory 12 and generate the second rotated image ROI2 from the other image IMG2. Note that in FIG. 16, the center of rotation O R3 is the point O in the lower right corner of the other image IMG2. D In this case, the rotation is realized by a combination of a rotation transformation and a translation. However, the center of rotation O R3 is the point O in the lower right corner of the other image IMG2. D The present invention is not limited to this, and may be applied to other points.

[0053] Next, the projective transformation in Fig. 17 will be described. In Fig. 17, the positions of pixels in the second intermediate image IIMG2 are expressed as a reference coordinate system D V -D S First, the second reference point O C In order to set the origin (0,0), the horizontal direction D of the third region RG3 S The additional trapezoidal image TZI2 can be generated by shifting the first target P1 to the left by the length a, creating a mirror image symmetrical with respect to the v axis, performing the same transformation as in FIG. 11 on the mirror image, and creating a mirror image symmetrical with respect to the v axis of the transformation result. B The second reference point O in the second rotated image ROI2 is set so that the part close to the work vehicle 1 (the eighth photographing object P8 in FIG. 17) is included in the additional trapezoidal image TZI2. C The vertical direction of D V Position v C Then, as in the case of FIG. 11, a search algorithm is used to determine the stretching factor t and the second reference point O C Horizontal direction D S Position of u C It is advisable to find the optimal solution.

[0054] However, the second reference point O in the second rotated image ROI2 C Position (uC ,v C The method of determining the position D of the fifth subject P5 and the sixth subject P6 is preferably one of the following methods to seamlessly combine the trapezoidal image TZI1 and the additional trapezoidal image TZI2. One method is to use the positions of the fifth subject P5 and the sixth subject P6. In the composite image CI, one of the fifth subject P5 and the sixth subject P6 is included in the trapezoidal image TZI1, and the other of the fifth subject P5 and the sixth subject P6 is included in the additional trapezoidal image TZI2. Therefore, in the composite image CI, the vertical direction D of the other subject is V The position of the one of the objects is in the vertical direction D V A second reference point O in the second rotated image ROI2 so as to substantially coincide with the position of C The vertical direction of D V Position v C In this way, the second reference point O C The vertical direction of D V Position v C When the first subject P1 is selected, the first subject P2 is moved backward D B In the image TZI, a portion close to the work vehicle 1 (for example, the eighth photographed object P8) is included in the additional trapezoidal image TZI2.

[0055] In addition, the first photographed object P1 displayed in the additional trapezoid image TZI2 is in the horizontal direction D of the composite image CI. S Since the other image IMG2 is substantially located at the center of the trapezoid image TZI1, the position of the symmetrical point of the position of the one photographing object displayed in the trapezoid image TZI1 with respect to the position of the first photographing object P1 displayed in the trapezoid image TZI1 is the position of the other photographing object displayed in the additional trapezoid image TZI2 in the composite image CI. Therefore, the horizontal distance D between the symmetrical point and the lower corner of the composite image CI (the lower right corner if the other image IMG2 is the second image IMGR, and the lower left corner if the other image IMG2 is the first image IMGL) is S The first distance L1 is the distance between the other object to be photographed and the second reference point O2 displayed in the additional trapezoid image TZI2. C and the lateral direction D SIf the other object to be photographed is, for example, the sixth object to be photographed P6, and the uv coordinates of the sixth object to be photographed P6 in the second intermediate image IIMG2 are expressed as (u6, v1) (the magnitude of the v coordinate is the same as the v coordinate v1 of the first object to be photographed P1), then

number

number

[0056] A second reference point O in the second rotated image ROI2 C Position (u C ,v C Another method for determining the second reference point O is to determine the second reference point O for the first to eighth photographing objects P1 to P8 that are displayed in the trapezoid image TZI1 and that are also displayed in the second rotated image ROI2 (the first to fourth photographing objects P1 to P4 in the example of FIG. 17). C The vertical direction of D V Position v C As an example, the second reference point O may be determined so that the first object P1 displayed in the trapezoid image TZI1 and the first object P1 displayed in the additional trapezoid image TZI2 are located at the same position in the composite image CI. C Position (u C ,vC ), u is determined using the same relationships as those in (Equation 3) to (Equation 6) described above. C , v C , t can be determined.

[0057] The parameters a and b for determining the size of the third region RG3 and the fourth region RG4, and the u determined as above C , v C , t are stored in the memory 12, and the hardware processor 11 executing the image processing program 15 may read these values ​​from the memory 12 to generate the additional trapezoidal image TZI2 from the second rotated image ROI2. Also, as described above, the second reference point O C The optimal position of is obtained by sequential search after rotation transformation. The third region RG3 shown in FIG. 15, FIG. 16, and FIG. 17 is the first reference point O determined in this way. A The position of the optimal first reference point O A Once this is determined, it can be said that the controller 10 (hardware processor 11 executing the image processing program 15) generates an additional trapezoidal image TZI2 by transforming the third region RG3 defined thereby (see Figures 15, 16, and 17) into a trapezoidal fourth region RG4 (see Figure 11).

[0058] From the above, it can be said that the controller 10 (the hardware processor 11 that executes the image processing program 15) has the following configuration. The controller 10 (the hardware processor 11 that executes the image processing program 15) detects whether the first photographed object P1 is substantially in the lateral direction D of the composite image CI. S The second reference point O, whose position does not change when the third region RG3 is transformed into the fourth region RG4, is determined to be located at the center of the C The controller 10 (hardware processor 11 executing the image processing program 15) is configured to generate an additional trapezoidal image TZI2 based on a position in an image (second rotated image ROI2) obtained from the other image IMG2 of the first image IMGL and the second image IMGR. B In the rear direction DB The second reference point O in the image (second rotated image ROI2) obtained from the other image IMG2 of the first image IMGL and the second image IMGR is determined so that a nearby portion (for example, the seventh photographing object P7) is included in the additional trapezoidal image TZI2. C , and is configured to generate an additional trapezoidal image TZI2 based on the position of

[0059] In other words, in order to make the composite image CI look natural as shown in FIG. 5, the controller 10 (the hardware processor 11 executing the image processing program 15) calculates the distance in the left and right direction D from the first photographing object P1. W The fifth and sixth photographed objects P5 and P6 are equidistant from each other in the left-right direction D of the composite image CI. W The vertical direction D of the composite image CI of the fifth photographed object P5 and the sixth photographed object P6 in the composite image CI is V The second reference point O in the image (second rotated image ROI2) obtained from the other image IMG2 is determined so that the position of the second reference point O in the image IMG2 substantially coincides with the position of the second rotated image ROI2. C This is because, even if the fifth photographing object P5 is displayed in one of the trapezoid image TZI1 and the additional trapezoid image TZI2 and the sixth photographing object P6 is displayed in the other of the trapezoid image TZI1 and the additional trapezoid image TZI2 as shown in FIG. 5, the vertical direction D of the composite image CI of the fifth photographing object P5 and the sixth photographing object P6 in the composite image CI is not changed. V The second reference point O is positioned so that the positions at C In this case, the position of the fifth photographed object P5 and the sixth photographed object P6 in the composite image CI is determined. V The term "substantially coincides with the position in the vertical direction D of the composite image CI between the fifth photographing object P5 and the sixth photographing object P6 in the composite image CI" means that a general operator who has seen the composite image CI V This means that the positions of the two images are recognized to be the same in the synthetic image CI (recognized as being at the same height in the synthetic image CI).

[0060] 13 and 14 can be used to generate the additional trapezoid image TZI2. That is, the controller 10 (hardware processor 11 executing the image processing program 15) performs projective transformation on the other image IMG2 so that the first arrangement direction AD1 and the second arrangement direction AD2 are substantially perpendicular to each other, and projects the projectively transformed image (the other trapezoidally corrected image TZO2) in the vertical direction D of the composite image CI of the third photographed object P3. V The position in the vertical direction D of the composite image CI of the fourth photographing object P4 V The center of rotation O is substantially the same as the position at R4 FIG. 18 is a diagram for explaining the projective transformation in this case, and FIG. 19 is a diagram for explaining the rotation transformation in this case. In FIG. 18, the positions of the pixels of one image IMG1 before the projective transformation are changed to the positions of the pixels of the lower right corner O D is expressed in the u'v' coordinate system with the origin (0,0).

[0061] The keystone correction in FIG. 18 is performed on the bottom right corner O of the other image IMG2. D In order to set the origin (0,0), the horizontal direction D of the other image IMG2 is S The other keystone correction image TZO2 can be generated by shifting the other image IMG2 to the left by the length c of the other image IMG2, creating a mirror image symmetrical with respect to the v axis, performing the same transformation as in FIG. 13 on the mirror image, and creating a mirror image symmetrical with respect to the v axis of the transformation result. S A stretching factor t can be determined by shifting the image IMG2 to the left by a length c and applying (Equation 8) to the mirror image symmetrical with respect to the v axis. The value t thus determined is stored in the memory 12, and the hardware processor 11 executing the image processing program 15 may read the value t from the memory 12 to generate the other keystone corrected image TZO2 from the other image IMG2.

[0062] FIG. 19 shows the other keystone corrected image TZO2 at the lower right corner point O D Rotation center O R4FIG. 19 shows the case where the third object P3 is rotated in the vertical direction DV and the fourth object P4 is rotated in the vertical direction DV. V The rotation angle θ4 is shown when the other keystone corrected image TZO2 is rotated so that the positions at the center of rotation O and the center of rotation P are substantially the same. This θ4 can be found by a well-known method from the inclination of the straight line P3P4 in the other keystone corrected image TZO2, so a detailed explanation is omitted. The value θ4 thus determined is stored in the memory 12, and the hardware processor 11 executing the image processing program 15 reads θ4 from the memory 12 and generates a second rotated keystone converted image TZRO2 from the other keystone corrected image TZO2. However, when the center of rotation O is R4 is the point O in the lower right corner of the other image IMG2. D The present invention is not limited to this, and may be applied to other points.

[0063] In addition, when generating the additional trapezoidal image TZI2 using the method of FIG. 18 and FIG. 19, the second reference point O C For example, the position of the first photographed object P1 is determined as follows: S (2) the first subject P1 is located in the rear direction D B The second reference point O in the second rotated trapezoidally transformed image TZRO2 is set so as to satisfy two conditions that the part (the seventh photographed object P7) close to the work vehicle 1 is included in the additional trapezoidal image TZI2. C Horizontal direction D S The position of the first object P1 displayed in the second rotated trapezoidal transformed image TZRO2 in the lateral direction D S From the position of the synthetic image CI, the horizontal direction D S The position shifted by half the length w / 2 is the second reference point O C Horizontal direction D S If the first reference point O is located at this position, condition (1) is satisfied. A The vertical direction of D V If the position of is determined, condition (2) is met.

[0064] However, the second reference point O in the second rotated image ROI2 C Position (u C ,v C The method of determining the second reference point O) is to use the positions of the fifth and sixth photographed objects P5 and P6 described in the example of FIG. 17 in order to seamlessly combine the trapezoidal image TZI1 and the additional trapezoidal image TZI2, or to use the photographed objects among the first to eighth photographed objects P1 to P8 that are displayed in the trapezoidal image TZI1 and that are also displayed in the second rotated trapezoidal conversion image TZRO2 (the first to fourth photographed objects P1 to P4 in the examples of FIGS. 18 and 19). C Position (u C ,v C 17 is different from the examples of FIGS. 18 and 19 in that the change in pixels before and after the keystone correction in FIG. 18 and the rotation transformation in FIG. 19 is smaller than the second reference point O C Therefore, when the size of the composite image CI and the coordinates in the composite image CI of any one of the first to fourth photographing objects P1 to P4 displayed by the trapezoid image TZI1 are known, the second reference point O for any one of the photographing objects in the other trapezoid-corrected image TZO2 is necessarily determined. C When the size of the composite image CI and the coordinates of either the fifth object P5 or the sixth object P6 displayed by the trapezoid image TZI1 in the composite image CI are known, the second reference point O for either of the objects in the other trapezoid-corrected image TZO2 is necessarily determined. C The relative positions of the

[0065] In this way, the second reference point O C When the first subject P1 is determined, the first subject P2 is B In the example of FIG. 18 and FIG. 19, the part closer to the work vehicle 1 in the rear direction DB than the first photographed object P1 (the seventh photographed object P7) is included in the additional trapezoidal image TZI2. S The length of the third region RG3 is a, and the length of the third region RG3 is D VThe length of the other image IMG2 is b, and the horizontal direction D S The length of the other image IMG2 is c, and the vertical direction D V If the length of the second reference point O in the second rotated trapezoidal transformed image TZRO2 is d, the length of the upper side of the fourth region RG4 is ab / d times t, which is obtained from the solution s of (Equation 8), and the shape of the second region RG2 can be determined by this. C Position (u C , v C When the second reference point O in the second rotated trapezoidally transformed image TZRO2 is determined, the position and shape of the fourth region RG4 is specified. C The position and size of the fourth region RG4 are stored in memory 12, and the hardware processor 11 executing the image processing program 15 may read these values ​​from memory 12 to generate an additional trapezoidal image TZI2 from the second rotated trapezoidally transformed image TZRO2.

[0066] The second reference point O shown above C The optimal position of the second reference point O shown in FIG. 15 will be obtained after keystone correction and rotation transformation. C The position of the second reference point O in the second rotated trapezoidally transformed image TZRO2 determined as described above. C The position of the second reference point O in the second rotated trapezoidally transformed image TZRO2 is obtained by performing an inverse transformation of the rotation transformation and an inverse transformation of the trapezoidal correction. C A second reference point O in the other image IMG2 corresponds to the position of C The shape of the third region RG3 shown in FIG. 15 can be considered to correspond to the position of the fourth region RG4 shown in FIG. 19, which is obtained by performing the inverse conversion of the keystone correction. C When the third region RG3 is determined, the controller 10 (the hardware processor 11 executing the image processing program 15) generates an additional trapezoidal image TZI2 by transforming the third region RG3 (see FIG. 15) defined thereby into a trapezoidal fourth region RG4 (see FIG. 19). V The length b of the composite image CI is the vertical direction D VThe length of the third region RG3 and the fourth region RG4 in the lateral direction D S The length of the fourth region RG4 in the horizontal direction D S The length of the composite image CI is the horizontal direction D S may be arbitrarily set to exceed half the length w / 2 of the first rotated trapezoidally transformed image TZRO2. In this case, the third region RG3 in the other image IMG2 corresponding to the fourth region RG4 in the second rotated trapezoidally transformed image TZRO2 is not necessarily rectangular. In this case, it can be said that the controller 10 (hardware processor 11 executing the image processing program 15) generates the additional trapezoidal image TZI2 by transforming the third region RG3 in the other image IMG2 corresponding to the fourth region RG4 in the second rotated trapezoidally transformed image TZRO2 into the trapezoidal fourth region RG4 shown in the second rotated trapezoidally transformed image TZRO2.

[0067] 20 and 21 are an example of a flowchart of the composite image generating method according to the first embodiment. This flowchart is a flowchart for the case where one image IMG1 shown in FIG. 9 is converted as shown in FIGS. 11 and 12, and the other image IMG2 shown in FIG. 15 is converted as shown in FIGS. 16 and 17. First, in step S1, in this generating method, F The optical axis Ax1 of the first camera 30L provided in the vicinity of the left rear corner of the vehicle body 2 of the work vehicle 1 faces in the left direction D. L From rear direction D B In step S2, the method is directed in a first direction D1 inclined toward the front direction D F The optical axis Ax2 of the second camera 30R, which is provided near the right rear corner of the vehicle body 2 and faces the traveling direction of the vehicle, is oriented in the right direction D. R From rear direction D BIn the generation method, the first camera 30L captures a first image IMGL. In the generation method, in step S3, the first camera 30L captures a first image IMGL. In the generation method, in step S4, the second camera 30R captures a second image IMGR. In step S5, the generation method, the controller 10 (hardware processor 11 executing the image processing program 15) acquires one image IMG1 of the first image IMGL and the second image IMGR.

[0068] Next, in step S10, in this generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) rotates one image IMG1 to obtain a first rotated image ROI1 as shown in FIG. 10. In the first rotated image ROI1, the vertical direction D in the composite image CI of the third photographing object P3 is V and the vertical direction D in the composite image CI of the fourth object P4 V The positions in the graph substantially coincide with each other.

[0069] In step S11, in the generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) calculates the vertical direction D V The position of the fourth object P4 in the composite image CI is in the vertical direction D V A first reference point O determined to substantially coincide with a position at A The vertical direction D in the first rotated image ROI1 V Position v A is obtained from the memory 12. Note that the first reference point O A The vertical direction of D V Position v A If the vertical direction D of the first rotated image ROI1 of the seventh photographing object P7 or the eighth photographing object P8 is not yet determined, V Using the position of the first reference point O A The vertical direction D in the first rotated image ROI1 V Position v A and storing the determined value in the memory 12.

[0070] In step S12, in the generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) performs a first alignment direction (vertical direction D V ) and the second alignment direction (horizontal direction D S ) is substantially orthogonal to the stretching coefficient t, which is obtained from the memory 12. If the stretching coefficient t has not yet been determined, the method may further include a process of determining the stretching coefficient t using a search algorithm as described above and storing the stretching coefficient t in the memory 12.

[0071] In step S13, in this generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) determines whether the first photographed object P1 is in the lateral direction D of the composite image CI. S A first reference point O determined to be substantially located at the center of A The horizontal direction D of the first rotated image ROI1 S Position of u A is obtained from the memory 12. Note that the first reference point O A The horizontal direction D of the first rotated image ROI1 S Position of u A If not yet determined, the first reference point O is found using the search algorithm as described above. A The horizontal direction D of the first rotated image ROI1 S Position of u A and storing the determined value in the memory 12.

[0072] In step S14, in this generation method, the controller 10 (hardware processor 11 executing image processing program 15) obtains a first intermediate image IIMG1 by extracting a first region RG1 in the first rotated image ROI1. In step S15, in this generation method, the controller 10 (hardware processor 11 executing image processing program 15) performs projective transformation on the first intermediate image IIMG1 using t obtained in step S12 to obtain a trapezoidal image TZI1.

[0073] In the generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) performs steps S12, S14, and S15 to generate a forward direction D F and backward D B The left-right direction D perpendicular to W The first camera 30L and the second camera 30R are disposed at substantially equal distances from each other in the rear direction D. B In the first arrangement direction (vertical direction D V ) and from the second photographing target P2, which is farther from the work vehicle 1 among the two photographing targets P1 to P2, in the left and right direction D W The third and fourth objects P3 and P4 are arranged in a second arrangement direction (horizontal direction D S ) are substantially perpendicular to each other, and the vertical direction D of the composite image CI V The higher you go, the more vertical D V The lateral direction D of the composite image CI perpendicular to S A trapezoidal image TZI1 can be generated by deforming the rectangular first region RG1 of an image (first rotated image ROI1) obtained from one image IMG1 of the first image IMGL and the second image IMGR into a trapezoidal second region RG2 so as to widen the width of the trapezoidal image TZI1.

[0074] Furthermore, since the trapezoidal image TZI1 generated in this manner is configured to cover the overlapping portion OVR where the same photographed object is displayed in the first image IMGL and the second image IMGR, as shown in Figures 7 and 8, in this generation method, by steps S12, S14 and S15, the controller 10 (hardware processor 11 executing the image processing program 15) can generate a trapezoidal image by deforming an image (first rotated image ROI1) obtained from one of the first image IMGL and the second image IMGR, image IMG1, into a trapezoid shape so as to cover the overlapping portion OVR where the same photographed object is displayed in the first image IMGL and the second image IMGR.

[0075] Next, in step S6 of FIG. 21, in this generation method, the controller 10 (hardware processor 11 executing the image processing program 15) acquires the other image IMG2 of the first image IMGL and the second image IMGR. In step S30, in this generation method, the controller 10 (hardware processor 11 executing the image processing program 15) rotates the other image IMG2 to obtain a second rotated image ROI2 as shown in FIG. 16. In the second rotated image ROI2, the vertical direction D in the composite image CI of the third photographing object P3 is rotated. V The position of the fourth object P4 in the composite image CI is in the vertical direction D V The positions in the graph substantially coincide with each other.

[0076] In step S31, in the generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) calculates a distance D from the first subject P1 in the rear direction. B The second reference point O is determined so that the portion close to the work vehicle 1 is included in the trapezoid image TZI1. C The vertical direction D in the second rotated image ROI2 V Position v C from the memory 12. As another method, in this generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) determines a second reference point O determined so that the position of one of the first to fourth shooting objects P1 to P4 displayed in the trapezoid image TZI1 and the position of the shooting object in the additional trapezoid image TZI2 are the same position. C The vertical direction of D V The position of the fifth object P5 and the sixth object P6 may be obtained from the memory 12. Alternatively, when one of the fifth object P5 and the sixth object P6 is included in the trapezoid image TZI1 and the other of the fifth object P5 and the sixth object P6 is included in the additional trapezoid image TZI2, the controller 10 (the hardware processor 11 executing the image processing program 15) obtains the vertical direction D of the other object in the composite image CI. V The position of the one of the objects is in the vertical direction D V A second reference point O determined to substantially coincide with the position ofC The vertical direction of D V Position v C The second reference point O C The vertical direction D in the second rotated image ROI2 V Position v C If not yet determined, the second reference point O is C The vertical direction D in the second rotated image ROI2 V Position v C and storing the determined value in the memory 12.

[0077] In step S32, in the generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) divides the composite image CI (additional trapezoid image TZI2) into the first arrangement direction (vertical direction D V ) and the second alignment direction (horizontal direction D S ) is substantially orthogonal to the stretching coefficient t, the stretching coefficient t being determined from the memory 12. If the stretching coefficient t has not yet been determined, the method may further include a process of determining the stretching coefficient t using a search algorithm as described above and storing the stretching coefficient t in the memory 12.

[0078] In step S33, in the generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) determines a second reference point O such that the position of the first photographed object P1 displayed in the additional trapezoid image TZI2 in the composite image CI substantially coincides with the position of the first photographed object P1 displayed in the trapezoid image TZI1. C The horizontal direction D of the second rotated image ROI2 S Position of u C is obtained from the memory 12. Note that the second reference point O C The horizontal direction D of the second rotated image ROI2 S Position of u C If not yet determined, the second reference point O is found using the search algorithm as described above. C The horizontal direction D of the second rotated image ROI2 S Position of u C and storing the determined value in the memory 12.

[0079] In step S34, in this generation method, the controller 10 (hardware processor 11 executing image processing program 15) obtains a second intermediate image IIMG2 by extracting a third region RG3 in the second rotated image ROI2. In step S35, in this generation method, the controller 10 (hardware processor 11 executing image processing program 15) performs projective transformation on the second intermediate image IIMG2 using t obtained in step S32 to obtain an additional trapezoidal image TZI2.

[0080] In the generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) performs the steps S32, S34, and S35 to generate a forward direction D F and backward D B The left-right direction D perpendicular to W The first camera 30L and the second camera 30R are disposed at substantially equal distances from each other in the rear direction D. B In the first arrangement direction (vertical direction D V ) and from the second photographing target P2, which is farther from the work vehicle 1 among the two photographing targets P1 to P2, in the left and right direction D W The third and fourth objects P3 and P4 are arranged in a second arrangement direction (horizontal direction D S ) are substantially perpendicular to each other, and the vertical direction D of the composite image CI V The higher you go, the more vertical D V The lateral direction D of the composite image CI perpendicular to S An additional trapezoidal image TZI2 can be generated by transforming the rectangular third region RG3 of the image (second rotated image ROI2) obtained from the other image IMG2 of the first image IMGL and the second image IMGR into a trapezoidal fourth region RG4 so as to widen the width of the trapezoidal image TZI2.

[0081] In step S50, in this generation method, the controller 10 (hardware processor 11 executing the image processing program 15) generates a composite image CI by superimposing the trapezoid image TZI1 on the additional trapezoid image TZI2. That is, in this generation method, the controller 10 (hardware processor 11 executing the image processing program 15) generates a composite image CI by superimposing the trapezoid image TZI1 on an image (additional trapezoid image TZI2) obtained from the other image IMG2 of the first image IMGL and the second image IMGR. In this generation method, the controller 10 (hardware processor 11 executing the image processing program 15) generates a composite image CI by superimposing the trapezoid image TZI1 on a superimposed image (additional trapezoid image TZI2) obtained from the other image IMG2 of the first image IMGL and the second image IMGR.

[0082] 22 and 23 are another example of a flowchart of the composite image generating method according to the first embodiment. This flowchart is a flowchart for the case where one image IMG1 shown in FIG. 9 is converted as shown in FIG. 13 and FIG. 14, and the other image IMG2 shown in FIG. 15 is converted as shown in FIG. 16 and FIG. 17. In FIG. 22 and FIG. 23, the same processes as those shown in FIG. 21 and FIG. 22 are denoted by the same reference numerals, and the description thereof will be omitted. In step S20 after step S5 in FIG. 22, in this generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) obtains one keystone corrected image TZO1 by transforming one image IMG1 so that the first arrangement direction AD1 and the second arrangement direction AD2 are substantially orthogonal to each other.

[0083] In step S21, in this generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) rotates one of the trapezoidally corrected images TZO1 to obtain a first rotated trapezoidally converted image TZRO1 as shown in FIG. 14. In the first rotated trapezoidally converted image TZRO1, the vertical direction D in the composite image CI of the third photographed object P3 is V and the vertical direction D in the composite image CI of the fourth object P4 VThe positions in the graph substantially coincide with each other.

[0084] In step S22, in the generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) calculates a distance D from the first subject P1 in the rear direction. B The first reference point O is determined so that a portion close to the work vehicle 1 is included in the trapezoid image TZI1. A The vertical direction D in the first rotated trapezoidally transformed image TZRO1 V Position v A is obtained from the memory 12. Note that the first reference point O A The vertical direction D in the first rotated trapezoidally transformed image TZRO1 V Position v A If the vertical direction D of the first rotated trapezoidally transformed image TZRO1 of the seventh photographed object P7 or the eighth photographed object P8 is not yet determined, V Using the position of the first reference point O A The vertical direction D in the first rotated trapezoidally transformed image TZRO1 V Position v A and storing the determined value in the memory 12.

[0085] In step S23, in this generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) determines whether the first photographed object P1 is in the lateral direction D of the composite image CI. S A first reference point O determined to be substantially located at the center of A The horizontal direction D in the first rotated trapezoidally transformed image TZRO1 S Position of u A is obtained from the memory 12. Note that the first reference point O A The horizontal direction D in the first rotated trapezoidally transformed image TZRO1 S Position of u A If the horizontal direction D in the first rotated trapezoidally transformed image TZRO1 of the first photographed object P1 has not yet been determined, S Position of the composite image CI and the lateral direction D S Using half the length w / 2, the first reference point O A The horizontal direction D in the first rotated trapezoidally transformed image TZRO1 S Position of uA and storing the determined value in the memory 12.

[0086] In step S24, in the generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) extracts the second region RG2 from the first rotated trapezoidal transformed image TZRO1 to obtain a trapezoidal image TZI1. F and backward D B The left-right direction D perpendicular to W The first camera 30L and the second camera 30R are disposed at substantially equal distances from each other in the rear direction D. B A first arrangement direction AD1 in which two subjects P1 to P2 are arranged at a distance from each other is arranged, and a left-right direction D from a second subject P2, which is farther from the work vehicle 1 among the two subjects P1 to P2, is arranged. W The second arrangement direction AD2 in which the third and fourth photographic objects P3 and P4, which are distant from each other, are arranged is substantially perpendicular to the vertical direction D of the composite image CI. V The higher you go, the more vertical D V The lateral direction D of the composite image CI perpendicular to S A trapezoidal image TZI1 can be generated by deforming the first region RG1 of an image (one image IMG1) obtained from one image IMG1 of the first image IMGL and the second image IMGR into a trapezoidal second region RG2 so as to widen the width of the trapezoidal image TZI1.

[0087] Furthermore, since the trapezoidal image TZI1 generated in this manner is configured to cover the overlapping portion OVR where the same photographed subject is displayed in the first image IMGL and the second image IMGR, as shown in Figures 7 and 8, in this generation method, by steps S20 and S24, the controller 10 (hardware processor 11 executing the image processing program 15) can generate a trapezoidal image TZI1 by deforming an image obtained from one of the first image IMGL and the second image IMGR, IMG1 (an image corresponding to the first region RG1 of the one image IMG1), into a trapezoid shape so as to cover the overlapping portion OVR where the same photographed subject is displayed in the first image IMGL and the second image IMGR.

[0088] Next, after step S6 in FIG. 23, in step S40, in this generation method, the controller 10 (hardware processor 11 executing the image processing program 15) obtains the other keystone corrected image TZO2 by deforming the other image IMG2 so that the first arrangement direction AD1 and the second arrangement direction AD2 are substantially perpendicular to each other. In step S41, in this generation method, the controller 10 (hardware processor 11 executing the image processing program 15) rotates the other keystone corrected image TZO2 to obtain a second rotated keystone converted image TZRO2 as shown in FIG. 19. In the second rotated keystone converted image TZRO2, the vertical direction D in the composite image CI of the third photographed object P3 is rotated so that the vertical direction D in the composite image CI of the third photographed object P3 is substantially perpendicular to the vertical direction D. V and the vertical direction D in the composite image CI of the fourth object P4 V The positions in the graph substantially coincide with each other.

[0089] In step S42, in the generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) calculates a distance D from the first subject P1 in the rear direction. B The second reference point O is determined so that the portion close to the work vehicle 1 is included in the additional trapezoid image TZI2. C The vertical direction D in the second rotated trapezoidally transformed image TZRO2 V Position v Cfrom the memory 12. As another method, in this generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) determines a second reference point O determined so that the position of one of the first to fourth shooting objects P1 to P4 displayed in the trapezoid image TZI1 and the position of the shooting object in the additional trapezoid image TZI2 are the same position. C The vertical direction D in the second rotated trapezoidally transformed image TZRO2 V The position of the fifth object P5 and the sixth object P6 may be obtained from the memory 12. Alternatively, when one of the fifth object P5 and the sixth object P6 is included in the trapezoid image TZI1 and the other of the fifth object P5 and the sixth object P6 is included in the additional trapezoid image TZI2, the controller 10 (the hardware processor 11 executing the image processing program 15) obtains the vertical direction D of the other object in the composite image CI. V The position of the one of the objects is in the vertical direction D V A second reference point O determined to substantially coincide with the position of C The vertical direction D in the second rotated trapezoidally transformed image TZRO2 V Position v C The second reference point O C The vertical direction D in the second rotated image ROI2 V Position v C If not yet determined, the second reference point O is C The vertical direction D in the second rotated trapezoidally transformed image TZRO2 V Position v C and storing the determined value in the memory 12.

[0090] In step S43, in this generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) determines whether the first photographed object P1 is in the lateral direction D of the composite image CI. S A second reference point O determined to be substantially located at the center of C The horizontal direction D in the second rotated trapezoidal transformed image TZRO2 S Position of u Cfrom the memory 12. As another method, in this generation method, the controller 10 (the hardware processor 11 executing the image processing program 15) determines a second reference point O determined so that the position of one of the first to fourth shooting objects P1 to P4 displayed in the trapezoid image TZI1 and the position of the shooting object in the additional trapezoid image TZI2 are the same position. C The horizontal direction D in the second rotated trapezoidal transformed image TZRO2 S The position of the fifth object P5 and the sixth object P6 may be obtained from the memory 12. Alternatively, when one of the fifth object P5 and the sixth object P6 is included in the trapezoid image TZI1 and the other of the fifth object P5 and the sixth object P6 is included in the additional trapezoid image TZI2, the controller 10 (the hardware processor 11 executing the image processing program 15) obtains the horizontal direction D of the other object in the composite image CI. S The position of the one of the objects is in the lateral direction D S A second reference point O determined to substantially coincide with the position of C The horizontal direction D in the second rotated trapezoidal transformed image TZRO2 S Position of u C The second reference point O C The horizontal direction D in the second rotated image ROI2 S Position v C If not yet determined, the second reference point O is C The horizontal direction D in the second rotated image ROI2 S Position v C and storing the determined value in the memory 12.

[0091] In step S44, in the generating method, the controller 10 (the hardware processor 11 executing the image processing program 15) extracts the fourth region RG4 from the second rotated trapezoidal transformed image TZRO2 to obtain a trapezoidal image TZI1. F and backward D B The left-right direction D perpendicular to WThe first camera 30L and the second camera 30R are disposed at substantially equal distances from each other in the rear direction D. B A first arrangement direction AD1 in which two subjects P1 to P2 are arranged at a distance from each other is arranged, and a left-right direction D from a second subject P2, which is farther from the work vehicle 1 among the two subjects P1 to P2, is arranged. W The second arrangement direction AD2 in which the third and fourth photographic objects P3 and P4, which are distant from each other, are arranged is substantially perpendicular to the vertical direction D of the composite image CI. V The higher you go, the more vertical D V The lateral direction D of the composite image CI perpendicular to S An additional trapezoidal image TZI2 can be generated by deforming the third region RG3 of the image (the other image IMG2) obtained from the other image IMG2 of the first image IMGL and the second image IMGR into a trapezoidal fourth region RG4 so as to widen the width of the trapezoidal image TZI2. <Actions and Effects of the First Embodiment>

[0092] The work vehicle 1 and the method for generating a composite image CI according to the first embodiment generate a trapezoidal image TZI1 by transforming a first region RG1 of an image obtained from one image IMG1 into a trapezoidal second region RG2 so that the first arrangement direction AD1 and the second arrangement direction AD2 are substantially perpendicular to each other, and generate a composite image CI by superimposing the trapezoidal image TZI1 on a superimposed image (for example, an additional trapezoidal image TZI2) obtained from the other image IMG2. Therefore, in order to obtain a wide angle of view behind the vehicle body, a composite image CI that allows the user to easily view the area behind the vehicle body can be generated by a simple algorithm from two cameras facing diagonally backward. <Second embodiment>

[0093] In the first embodiment, the first reference point O A When the position of the second reference point O is determined, it is assumed that the first region RG1 of the first rotated image ROI1 does not include a non-display region (a region shown in black in the image), and the second region RG2 of the first rotated trapezoidally transformed image TZRO1 does not include a non-display region. CWhen the position is determined, it is assumed that no non-display area is included in the third region RG3 of the second rotated image ROI2, and no non-display area is included in the fourth region RG4 of the second rotated trapezoidally transformed image TZRO2.

[0094] However, the first reference point O determined by the above-mentioned search algorithm A When the first region RG1 of the first rotated image ROI1 selected based on the position of the first region RG1 includes a non-display area as shown in FIG. 24, the first rotated image ROI1 may be enlarged to such an extent that the non-display area is not included. This enlargement ratio is determined by the horizontal direction D of the first region RG1. S The length a of the first region RG1 is divided by the shorter length LG1 of the upper or lower side of the first region RG1 other than the non-display region. V and the horizontal direction D S The image is enlarged at the corresponding magnification ratio.

[0095] In addition, the second reference point O determined by the above-mentioned search algorithm C When the third region RG3 of the second rotated image ROI2 selected based on the position of the third region RG3 includes a non-display area as shown in FIG. 25, the second rotated image ROI2 may be enlarged to such an extent that the non-display area is not included. This enlargement ratio is determined by the horizontal direction D of the third region RG3. S The length a of the third region RG3 is divided by the shorter length LG2 of the upper or lower side of the third region RG3 other than the non-display region. V and the horizontal direction D S If the first camera 30L and the second camera 30R are the same type of camera, have the same focal length, and are arranged in symmetrical positions and orientations on the work vehicle 1, it is advisable to match the magnification ratio of the first rotated image ROI1 and the magnification ratio of the second rotated image ROI2.

[0096] When the first rotated image ROI1 is enlarged in this manner, a stretching factor t and a first reference point O are set for the enlarged first rotated image ROI1.A Position (u A ,v A ) may be recalculated. When the second rotated image ROI2 is enlarged in this way, the stretching factor t and the second reference point O may be recalculated for the enlarged second rotated image ROI2. C Position (u C ,v C ) may be recalculated. By using the parameters recalculated in this manner, the controller 10 (hardware processor 11 executing the image processing program 15) is configured to enlarge the first rotated image ROI1 so that there is no non-display area in the image (first intermediate image IIMG1) corresponding to the first region RG1 of the first rotated image ROI1, and to perform projective transformation on the image (enlarged first intermediate image IIMG1) corresponding to the first region RG1 of the enlarged first rotated image ROI1 to generate a trapezoidal image TZI1. The controller 10 (hardware processor 11 executing the image processing program 15) is configured to enlarge the second rotated image ROI2 so that there is no non-display area in the image (second intermediate image IIMG2) corresponding to the third region RG3 of the second rotated image ROI2, and to perform projective transformation on the image (enlarged second intermediate image IIMG2) corresponding to the third region RG3 of the enlarged second rotated image ROI2 to generate an additional trapezoidal image TZI2.

[0097] However, the first reference point O determined based on the position of the subject displayed in the first rotated trapezoidally transformed image TZRO1 as described above A When the second region RG2 of the first rotated trapezoidally converted image TZRO1 selected based on the position of the first region RG2 includes a non-display area as shown in FIG. 26, the first rotated trapezoidally converted image TZRO1 may be enlarged to such an extent that the non-display area is not included. This enlargement ratio is set by dividing the upper side of the second region RG2 by the horizontal direction D S The length t*ab / d of the upper side of the second region RG2 is the horizontal direction D of the part other than the non-display area. S The value divided by the length LG3 and the horizontal direction D of the lower side of the second region RG2 S The length a of the second region RG2 is the horizontal direction D of the part other than the non-display area at the bottom side of the second region RG2. SWhen the first rotated trapezoidally transformed image TZRO1 is enlarged, the vertical direction D V and the horizontal direction D S The image is enlarged at the corresponding magnification ratio.

[0098] In addition, the second reference point O determined based on the position of the subject displayed in the second rotated trapezoidally transformed image TZRO2 as described above C When the fourth region RG4 of the second rotated trapezoidally converted image TZRO2 selected based on the position of the fourth region RG4 includes a non-display area as shown in FIG. 27, the second rotated trapezoidally converted image TZRO2 may be enlarged to such an extent that the non-display area is not included. This enlargement ratio is set by dividing the fourth region RG4 by the horizontal direction D S The length t*ab / d of the fourth region RG4 is the horizontal direction D of the part other than the non-display area. S The value divided by the length LG5 and the horizontal direction D of the lower side of the fourth region RG4 S The length a of the fourth region RG4 is the horizontal direction D of the part other than the non-display area. S When the second rotated trapezoidally transformed image TZRO2 is enlarged, the vertical direction D V and the horizontal direction D S If the first camera 30L and the second camera 30R are the same type of camera, have the same focal length, and are arranged in positions and orientations symmetrical to each other on the work vehicle 1, it is advisable to match the magnification ratio of the first rotated trapezoidally transformed image TZRO1 and the magnification ratio of the second rotated trapezoidally transformed image TZRO2.

[0099] When the first rotated trapezoidally transformed image TZRO1 is enlarged in this manner, a first reference point O is set for the enlarged first rotated trapezoidally transformed image TZRO1. A Position (u A ,v A ) may be recalculated. When the second rotated trapezoidally transformed image TZRO2 is enlarged in this manner, the second reference point O C Position (u C ,vC ) may be recalculated. By using the parameters recalculated in this manner, the controller 10 (hardware processor 11 executing the image processing program 15) is configured to enlarge the first rotated trapezoidally transformed image TZRO1 so that there is no non-display area in the second region RG2 of the first rotated trapezoidally transformed image TZRO1, and extract the second region RG2 from the enlarged first rotated trapezoidally transformed image TZRO1 to generate a trapezoidal image TZI1. The controller 10 (hardware processor 11 executing the image processing program 15) is configured to enlarge the second rotated trapezoidally transformed image TZRO2 so that there is no non-display area in the fourth region RG4 of the second rotated trapezoidally transformed image TZRO2, and extract the fourth region RG4 from the enlarged second rotated trapezoidally transformed image TZRO2 to generate an additional trapezoidal image TZI2.

[0100] 28 and 29 are an example of a flowchart of a composite image generating method according to the second embodiment. This flowchart is obtained by adding a process for enlarging the first rotated image ROI1 and the second rotated image ROI2 to the flowcharts of FIGS. 20 and 21. In this flowchart, the same processes as those in the flowcharts of FIGS. 20 and 21 are given the same reference numerals, and their explanations will be omitted. After step S10 in FIG. 28, in step S16, the controller 10 (hardware processor 11 executing the image processing program 15) enlarges the first rotated image ROI1 in the vertical direction D at the enlargement ratio determined as described above. V and the horizontal direction D S After step S30 in FIG. 29, in step S36, the controller 10 (the hardware processor 11 executing the image processing program 15) rotates the second rotated image ROI2 in the vertical direction D at the enlargement ratio determined as above. V and the horizontal direction D S to expand.

[0101] Figures 30 and 31 are another example of a flowchart of the composite image generating method according to the second embodiment. This flowchart adds processing for enlarging the first rotated trapezoidally transformed image TZRO1 and the second rotated trapezoidally transformed image TZRO2 to the flowcharts of Figures 22 and 23. In this flowchart, the same processing as in the flowcharts of Figures 22 and 23 is given the same reference numerals, and description thereof will be omitted. After step S21 in Figure 30, in step S25, the controller 10 (hardware processor 11 executing the image processing program 15) enlarges the first rotated trapezoidally transformed image TZRO1 in the vertical direction D at the enlargement ratio determined as described above. V and the horizontal direction D S After step S41 in FIG. 31, in step S45, the controller 10 (the hardware processor 11 executing the image processing program 15) enlarges the second rotated trapezoidally transformed image TZRO2 in the vertical direction D at the enlargement ratio determined as above. V and the horizontal direction D S to expand. <Actions and Effects of the Second Embodiment>

[0102] In the work vehicle 1 and the method for generating a composite image CI according to the second embodiment, when the trapezoid image TZI1 generated by the method according to the first embodiment includes a non-display area, the original image for generating the trapezoid image TZI1 is enlarged. Therefore, a composite image CI that makes it easier for the user to view the area behind the vehicle body can be generated by a simple algorithm. <Modifications of all embodiments>

[0103] The above-described composite image CI is rectangular, but may be trapezoidal or another shape in accordance with the shape of the monitor 53. A Position (u A ,v A ), second reference point O C Position (u C ,v CThe method of determining the above-mentioned parameters is not limited to the method specified in the above embodiment, and may be a method gradually modified from a provisionally determined parameter. In addition, the process of step S6 and steps S40 to S50 in FIG. 23 may be executed after steps S1 to S15 in FIG. 20. The process of step S6 and steps S30 to S50 in FIG. 21 may be executed after steps S1 to S5 and steps S20 to S24 in FIG. 22. The process of step S6 and steps S40 to S50 in FIG. 31 may be executed after steps S1 to S15 including step S16 in FIG. 28. The process of step S6 and steps S30 to S50 in FIG. 29 including step S36 may be executed after steps S1 to S5 and steps S20 to S24 including step S25 in FIG. 30.

[0104] In the above embodiment, a method for obtaining a desired composite image CI has been shown as simplified as possible, but the above composite image CI may be obtained by changing the order of rotation transformation, trapezoid transformation, enlargement, and cutting out one image and the other image. A Position (u A ,v A ), second reference point O C Position (u C ,v C The determination of the first reference point O is not limited to after the rotation transformation and trapezoid transformation, but can be done before the rotation transformation and trapezoid transformation. A Position (u A ,v A ), second reference point O C Position (u C ,v C ) is provisionally determined, and the first reference point O is created after rotation and trapezoid transformation. A Position (u A ,v A ), second reference point O C Position (u C ,v C ) may be finally positioned.

[0105] In this application, the term "comprises" and its derivatives are open-ended terms that describe the presence of elements and do not exclude the presence of other elements not listed. This also applies to the terms "have", "include" and their derivatives.

[0106] The terms "member," "part," "element," "body," and "structure" may have multiple meanings, such as a single part or multiple parts.

[0107] Ordinal numbers such as "first" and "second" are merely terms used to identify components and do not have any other meaning (such as a particular order). For example, the presence of a "first element" does not imply the presence of a "second element," and the presence of a "second element" does not imply the presence of a "first element."

[0108] Words expressing degree, such as "substantially," "about," and "approximately," may mean a reasonable deviation that does not significantly change the final result, unless otherwise specified in the embodiment. All numerical values ​​described in this application may be interpreted to include words such as "substantially," "about," and "approximately."

[0109] In this application, the phrase "at least one of A and B" should be interpreted as including A only, B only, and both A and B.

[0110] It is apparent that various changes and modifications of the present invention are possible in light of the above disclosure, and therefore, the present invention may be practiced otherwise than as specifically disclosed herein without departing from the spirit and scope of the present invention.

Claims

1. A pair of traveling devices each configured to move in a forward direction and a rearward direction; A vehicle body provided between the pair of traveling devices in a left-right direction perpendicular to the front direction and the rear direction; A first camera provided near a left rear corner of the vehicle body; A second camera provided near a right rear corner of the vehicle body; a controller configured to generate a composite image by combining a portion of a first image captured by the first camera and a portion of a second image captured by the second camera; a monitor configured to display the composite image; and A work vehicle comprising: an optical axis of the first camera extends in a first direction inclined from a left direction toward the rear direction; an optical axis of the second camera extends in a second direction inclined from the right direction toward the rear direction; In the composite image, the further upward in the vertical direction, the more the subject located in the rear direction is displayed, the controller is configured to generate a trapezoidal image by deforming a first region of an image obtained from one of the first image and the second image into a trapezoidal second region such that a width in a horizontal direction perpendicular to the vertical direction increases toward the top in the vertical direction, the trapezoidal image being generated such that a first arrangement direction in which two subjects to be photographed that are disposed at substantially equal distances from the first camera and the second camera in the left-right direction and spaced apart from each other in the rear direction are arranged is substantially perpendicular to a second arrangement direction in which a third subject to be photographed and a fourth subject to be photographed that are distant in the left-right direction from a second subject to be photographed that is farther from the work vehicle among the two subjects to be photographed are arranged; and the controller is configured to superimpose the trapezoidal image on a superimposed image obtained from the other of the first image and the second image to generate the composite image. Work vehicle.

2. the controller is configured to generate the trapezoidal image by projecting an image corresponding to the first region of a first rotated image obtained by rotating one of the first image and the second image such that a position of the third photographed object in the composite image in the vertical direction substantially coincides with a position of the fourth photographed object in the composite image in the vertical direction. A work vehicle as claimed in claim 1.

3. the controller is configured to enlarge the first rotated image so that no non-display area exists in an image corresponding to the first area of ​​the first rotated image, and to perform projective transformation on the image corresponding to the first area of ​​the enlarged first rotated image to generate the trapezoidal image. A work vehicle as claimed in claim 2.

4. the controller is configured to perform projective transformation on one of the first image and the second image so that the first arrangement direction and the second arrangement direction are substantially perpendicular to each other, and to generate the trapezoidal image by extracting the second region from a first rotated trapezoidally transformed image obtained by rotating the projectively transformed image so that a position of the third photographed subject in the vertical direction in the composite image of the fourth photographed subject substantially coincides with a position of the fourth photographed subject in the vertical direction. A work vehicle as claimed in claim 1.

5. the controller is configured to enlarge the first rotated trapezoidally transformed image so that no non-display area exists in the second area of ​​the first rotated trapezoidally transformed image, and extract the second area from the enlarged first rotated trapezoidally transformed image to generate the trapezoidal image. A work vehicle as claimed in claim 4.

6. the controller is configured to generate the trapezoidal image based on a position in an image obtained from one of the first image and the second image of a first reference point whose position is not changed when the first region is transformed into the second region, the first reference point being determined so that a first photographing target of the two photographing targets that is closer to the work vehicle in the rear direction is substantially positioned at the center of the lateral direction of the composite image; A work vehicle according to any one of claims 2 to 5.

7. the controller is configured to generate the trapezoidal image based on a position of the first reference point in an image obtained from one of the first image and the second image, the position being determined so that a portion closer to the work vehicle than the first photographed target in the rear direction is included in the trapezoidal image. A work vehicle as set forth in claim 6.

8. the controller is configured to generate, as the superimposed image, an additional trapezoidal image obtained by transforming a third region of an image obtained from the other of the first image and the second image into a trapezoidal fourth region such that the first arrangement direction and the second arrangement direction are substantially perpendicular to each other and the width in the horizontal direction increases toward the top in the vertical direction; A work vehicle according to any one of claims 2 to 5.

9. the controller is configured to generate the additional trapezoidal image by projecting an image corresponding to the third region of a second rotated image obtained by rotating the other of the first image and the second image such that a position of the third photographed object in the composite image in the vertical direction substantially coincides with a position of the fourth photographed object in the composite image in the vertical direction. A work vehicle according to claim 8.

10. the controller is configured to enlarge the second rotated image so that no non-display area exists in an image corresponding to the third area of ​​the second rotated image, and to perform projective transformation on an image corresponding to the third area of ​​the enlarged second rotated image to generate the additional trapezoidal image. A work vehicle as claimed in claim 9.

11. the controller is configured to perform projective transformation on the other of the first image and the second image so that the other has a trapezoidal shape, so that the first arrangement direction and the second arrangement direction are substantially perpendicular to each other, and to generate the additional trapezoidal image by extracting the fourth region from a second rotated trapezoidally transformed image obtained by rotating the projectively transformed image so that a position of the composite image of the third photographed subject in the vertical direction substantially coincides with a position of the composite image of the fourth photographed subject in the vertical direction. A work vehicle according to claim 8.

12. the controller is configured to enlarge the second rotated trapezoidally transformed image so that no non-display area exists in the fourth area of ​​the second rotated trapezoidally transformed image, and extract the fourth area from the enlarged second rotated trapezoidally transformed image to generate the additional trapezoidal image. A work vehicle as claimed in claim 11.

13. the controller is configured to generate the additional trapezoidal image based on a position in an image obtained from the other of the first image and the second image of a second reference point whose position is not changed when the third region is transformed into the fourth region, the second reference point being determined so that a first photographing target of the two photographing targets that is closer to the work vehicle in the rear direction is substantially positioned at the center of the lateral direction of the composite image. A work vehicle according to claim 8.

14. the controller is configured to generate the additional trapezoidal image based on a position of the second reference point in an image obtained from the other of the first image and the second image, the position being determined so that a portion closer to the work vehicle in the rear direction than the first photographed target is included in the additional trapezoidal image. A work vehicle according to claim 13.

15. A driver's seat is provided on the vehicle body, The pair of traveling devices are A first traveling device disposed on the left side of the vehicle body; A second traveling device disposed on the right side of the vehicle body; Including, The first camera is provided between the driver's seat and the first traveling device in the left-right direction, The second camera is provided between the driver's seat and the second traveling device in the left-right direction, The first camera is provided rearward of a rear end of the first traveling device, The second camera is provided rearward of a rear end of the second traveling device. A work vehicle according to any one of claims 1 to 5.

16. a pair of arm mechanisms each provided between the pair of traveling devices and the driver's seat in the left-right direction and supporting a working tool; Further equipped with The pair of arm mechanisms include: a first arm mechanism located between the first traveling device and the driver's seat in the left-right direction and rotatably supported by the vehicle body via a first fulcrum shaft; a second arm mechanism located between the second traveling device and the driver's seat in the left-right direction and rotatably supported by the vehicle body via a second fulcrum shaft, the first camera is provided between the driver's seat and the first arm mechanism in the left-right direction, the second camera is provided between the driver's seat and the second arm mechanism in the left-right direction, the first camera is provided rearward of the first fulcrum shaft, The second camera is provided rearward of the second fulcrum axis. A work vehicle as claimed in claim 15.

17. The optical axis of a first camera provided near a left rear corner of a vehicle body of the work vehicle facing in a forward traveling direction of a traveling device of the work vehicle is directed in a first direction inclined from the left direction toward the rear direction, an optical axis of a second camera provided in the vicinity of a right rear corner of the vehicle body facing the forward traveling direction is directed in a second direction inclined from the right direction toward the rear direction; capturing a first image with the first camera; capturing a second image with the second camera; a first arrangement direction in which two subjects to be photographed are arranged at substantially equal distances between the first camera and the second camera in a left-right direction perpendicular to the forward direction and the rearward direction and spaced apart from each other in the rearward direction are arranged, and a second arrangement direction in which a third subject to be photographed and a fourth subject to be photographed that are spaced apart in the left-right direction from a second subject to be photographed that is farther from the work vehicle out of the two subjects to be photographed are arranged, are substantially perpendicular to each other, and a trapezoidal image is generated by deforming a first region of an image obtained from one of the first image and the second image into a trapezoidal second region such that the width of the composite image in the horizontal direction perpendicular to the vertical direction increases the further upward in the vertical direction of the composite image; generating the composite image by superimposing the trapezoid image on a superimposed image obtained from the other of the first image and the second image; Including, In the composite image, the further upward in the vertical direction, the more the subject located in the rear direction is displayed. A synthetic image generation method.

18. A pair of traveling devices each configured to move in a forward direction and a rearward direction; a vehicle body provided between the pair of traveling devices in a left-right direction perpendicular to the front direction and the rear direction and rotatably supporting the pair of traveling devices; A first camera provided near a left rear corner of the vehicle body; A second camera provided near a right rear corner of the vehicle body; a controller configured to generate a composite image by combining a portion of a first image captured by the first camera and a portion of a second image captured by the second camera; a monitor configured to display the composite image; and A work vehicle comprising: an optical axis of the first camera extends in a first direction inclined from a left direction toward the rear direction; an optical axis of the second camera extends in a second direction inclined from the right direction toward the rear direction; the controller is configured to generate a trapezoidal image by deforming an image obtained from one of the first image and the second image into a trapezoid shape so as to cover an overlapping portion displayed together in the first image and the second image by using the first camera and the second camera to capture an image of the same subject, and to superimpose the trapezoidal image on the image obtained from the other of the first image and the second image to generate the composite image. Work vehicle.

19. A first camera is provided in a left rear corner of a vehicle body of the work vehicle and faces a forward traveling direction of the travel device of the work vehicle, and the optical axis of the first camera is directed in a first direction inclined from the left direction toward the rear direction; a second camera provided in the vicinity of a right rear corner of the vehicle body facing the forward traveling direction, the optical axis of the second camera being oriented in a second direction inclined from the right direction toward the rear direction; capturing a first image with the first camera; capturing a second image with the second camera; generating a trapezoidal image by deforming an image obtained from one of the first image and the second image into a trapezoidal shape so as to cover an overlapping portion in which the same photographic subject is displayed in the first image and the second image; generating a composite image by superimposing the trapezoid image on an image obtained from the other of the first image and the second image; A method for generating a synthetic image, comprising:

Images