Work vehicle and control program for work vehicle

Abstract

To provide a work vehicle that can be positioned at a location where a work machine can be coupled at low cost.SOLUTION: A work vehicle that can be coupled with a work machine, includes: an imaging device; and a control device that controls the driving of the work vehicle on the basis of images captured by the imaging device. The control device detects feature quantities of the work machine from an image obtained by imaging the work machine, drives the work vehicle on the basis of the feature quantities to position the work vehicle at a predetermined position where the imaging device can image an orientation target provided on the work machine, and drives the work vehicle to a position where it can be coupled with the work machine on the basis of an image obtained by imaging the orientation target provided on the work machine using the imaging device at the predetermined position.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a work vehicle and a control program for a work vehicle.

Background Art

[0002] Work implements that are towed by a work vehicle such as a tractor or attached to a work vehicle to perform various operations are known. There are many types of work implements. As an example, work implements for performing fertilization, tillage, pesticide spraying, seeding, etc. can be mentioned.

[0003] The work implement is selected according to the content of the agricultural work and attached to the work vehicle. At that time, a connection operation between the work vehicle and the work implement is required. This connection operation is complicated, and there is concern about the occurrence of accidents during the operation. As a technique for dealing with this problem, the technique of Patent Document 1 is known. Patent Document 1 describes a connecting member that makes the connection between a tractor and a work implement easier.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] A method of connecting a work implement to a work vehicle can be considered by mounting a measuring device on the work vehicle and automatically controlling the work vehicle based on the results of measuring the positional relationship and the attitude relationship between the work vehicle and the work implement. Here, considering practicality, the use of expensive measuring devices is not a good strategy.

[0006] The present invention has been made under such circumstances, and an object thereof is to provide a work vehicle that can be positioned at a low cost at a position where a work implement can be connected, and a control program for the work vehicle.

Means for Solving the Problems

[0007] The work vehicle of the present invention is a work vehicle that can be connected to a work implement, and includes a photographing device and a control device that controls the driving of the work vehicle based on an image photographed by the photographing device. The control device detects a feature amount of the work implement from an image of the work implement photographed, drives the work vehicle based on the feature amount, positions the work vehicle at a predetermined position where the photographing device can photograph a calibration target provided on the work implement, and drives the work vehicle to a position where it can be connected to the work implement based on an image of the calibration target provided on the work implement photographed using the photographing device at the predetermined position. It is a work vehicle.

Advantages of the Invention

[0008] The work vehicle of the present invention and the control program of the work vehicle have the effect of being able to position the work vehicle at a low cost at a position where it can be connected to the work implement.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the work vehicle system 1000 according to the present embodiment will be described in detail with reference to FIGS. 1 to 9. FIG. 1 is a diagram schematically showing the work vehicle system 1000 of the present embodiment.

[0011] As shown in FIG. 1, the work vehicle system 1000 includes a tractor 100 as a work vehicle and a work implement 200. In FIG. 1, the front-rear direction of the tractor 100 is defined as the Y-axis direction, the front of the tractor 100 is the -Y direction, and the rear is the +Y direction. Also, the left-right direction of the tractor is the X-axis direction, and the direction orthogonal to the X-axis and Y-axis is the Z-axis direction.

[0012] At the rear of the tractor 100, a camera 140 as a photographing device is provided. The camera 140 is set to be able to photograph the rear of the tractor 100. In the present embodiment, the camera 140 is arranged directly behind on the central axis of the tractor 100. The position and orientation of the camera 140 on the tractor 100 are measured in advance and are known. The camera 140 is a single-eye digital still camera and continuously takes still images. The shooting interval is set to 0.25 seconds to 1 second. Note that the camera 140 may be a camera that shoots videos, and the frame images constituting the videos may be used as still images. Note that the camera 140 may be a relatively inexpensive camera such as an industrial camera or a web camera in addition to the digital still camera. Note that the camera 140 is calibrated in advance to obtain internal calibration elements (correction parameters for correcting optical system distortion, etc.).

[0013] The working machine 200 is a working machine that performs, for example, fertilization, tillage, pesticide spraying, seeding, etc. The working machine 200 is selected according to the content of the agricultural work and is attached to the tractor 100. When attaching the working machine 200 to the tractor 100, a connection operation is required. However, in the present embodiment, by controlling the operation of the tractor 100, the working machine 200 is automatically connected to the tractor 100.

[0014] On the rear part (+Y side) of the tractor 100, a tractor-side hitch frame 110 is provided via a link mechanism 149. Also, on the front (-Y side) of the working machine 200, a working machine-side hitch frame 210 is provided. The tractor 100 and the working machine 200 are connected by the tractor-side hitch frame 110 and the working machine-side hitch frame 210. Note that in FIG. 1, the case where the working machine 200 is provided with wheels is illustrated, but it is not limited thereto, and the working machine 200 may be connected to the rear part of the tractor 100 in a form that floats in the air without wheels.

[0015] Figure 2 is a perspective view showing the rear part of the tractor 100. As shown in Figure 2, a PTO shaft (Power take off) 148 and a link mechanism 149 are provided at the rear part of the tractor 100.

[0016] The PTO shaft 148 extends in the substantially Y-axis direction and is connected to the spline shaft 117 (see Figure 3(a)) of the tractor-side hitch frame 110 via a telescopic joint (not shown).

[0017] The link mechanism 149 includes a pair of lower links 141, 142, an upper link 143, and a pair of lift rods 144, 145.

[0018] The pair of lower links 141, 142 extend in the substantially Y-axis direction in Figure 2, and one end (-Y side end) thereof is connected to the rear part of the tractor 100. The lower links 141, 142 are rotatable about the X-axis with respect to the tractor 100 at one end (-Y side end).

[0019] The upper link 143 extends in the substantially Y-axis direction in Figure 2, and one end (-Y side end) thereof is connected to the tractor 100. The upper link 143 is rotatable about the X-axis with respect to the rear part of the tractor 100 at one end (-Y side end). Further, the upper link 143 is capable of being extended and contracted by power.

[0020] One end of each of the lift rods 144, 145 is connected to the lower links 141, 142, and the other end is connected to the rear part of the tractor 100. Also, a joint portion is provided between one end and the other end. A driving force generated by a driving device (not shown) acts on the other ends of the lift rods 144, 145, and the driving force is transmitted to the lower links 141, 142 via the lift rods 144, 145, so that the lower links 141, 142 rotate about the X-axis.

[0021] (Tractor-side hitch frame 110) FIG. 3(a) is a perspective view showing the tractor-side hitch frame 110 as viewed from the -Y side (the side attached to the tractor 100), and FIG. 3(b) is a perspective view showing the tractor-side hitch frame 110 as viewed from the +Y side (the side to which the work implement 200 is connected).

[0022] The tractor-side hitch frame 110 includes a substantially triangular frame structure 116, a pair of lower support portions 111 and 112 provided at the lower end portion of the frame structure 116, an upper support portion 190 provided on the -Y side of the frame structure 116, and a spline shaft 117 extending in the Y-axis direction.

[0023] The pair of lower support portions 111 and 112 are provided at the lower end portion of the frame structure 116 in a state of protruding toward the -Y side, and are respectively connected to the pair of lower links 141 and 142 in FIG. 2.

[0024] The upper support portion 190 has an elongated hole and a plurality of round holes. The tip of the upper link 143 in FIG. 2 is connected to one of these holes via a horizontal rod (not shown). The upper support portion 190 has a plurality of holes in order to be connectable to tractors of different sizes and types. Therefore, it is selected which hole of the upper support portion 190 to use according to the type and size of the tractor.

[0025] By adjusting the degree of expansion and contraction of the upper link 143 at the rear of the tractor 100, the front-rear inclination of the tractor-side hitch frame 110 can be adjusted. Also, by adjusting the inclination of the lower links 141 and 142 at the rear of the tractor 100, the vertical position of the tractor-side hitch frame 110 with respect to the tractor 100 can be adjusted.

[0026] The spline shaft 117 is connected to the PTO shaft 148 provided at the rear part of the tractor 100 via a coupling. The spline shaft 117 has axially extending teeth formed on its outer periphery. The tractor-side hitch frame 110 is provided with a drive mechanism for driving the spline shaft 117 axially forward and backward (-Y direction and +Y direction). Due to this forward and backward drive, the spline shaft 117 is adapted to be coupled to the spline sleeve shaft 212 of the spline sleeve structure on the side of the working machine 200 (see FIGS. 4(a) and 4(b)). Thereby, the connection between the PTO shaft 148 of the tractor 100 and the PIC shaft (not shown) of the working machine 200 is made. The PIC shaft is the shaft on the side of the working machine 200 that receives the driving force from the tractor 100, which is called the power input connect shaft.

[0027] Also, as shown in FIG. 3(b), the tractor-side hitch frame 110 is provided with a hook 119. The hook 119 is hooked on an opening 218 provided in the working machine-side hitch frame 210 of FIGS. 4(a) and 4(b), whereby the coupled state between the tractor-side hitch frame 110 and the working machine-side hitch frame 210 is locked.

[0028] Also, a connector 160 is provided at the upper end portion (+Z side end portion) of the tractor-side hitch frame 110. This connector 160 is adapted to be connected to a receiving-side connector (not shown) of the working machine-side hitch frame 210.

[0029] (Working machine-side hitch frame 210) FIG. 4(a) is a perspective view showing the state of the working machine-side hitch frame 210 as viewed from the -Y side (the side attached to the tractor-side hitch frame 110), and FIG. 4(b) is a perspective view showing the state of the working machine-side hitch frame 210 as viewed from the +Y side (the side of the working machine 200).

[0030] The work machine side hitch frame 210 includes a substantially triangular frame structure 211, a pair of lower fixing members 214 and 215 provided at the lower end of the frame structure 211, an upper fixing member 213 provided on the +Y side of the frame structure 211, a spline slip shaft 212 provided for the frame structure 211 via a bearing structure, and a calibration target portion 220 provided above the frame structure 211.

[0031] The pair of lower fixing members 214 and 215 and the upper fixing member 213 are parts for fixing the work machine side hitch frame 210 to the work machine 200.

[0032] As described above, the spline shaft 117 of the tractor side hitch frame 110 is connected to the spline slip shaft 212. Also, the end of the spline slip shaft 212 on the work machine 200 side is connected to a PIC shaft (not shown) of the work machine 200.

[0033] When connecting the tractor side hitch frame 110 and the work machine side hitch frame 210, alignment in the horizontal direction is performed between the frame structure 116 of the tractor side hitch frame 110 and the frame structure 211 of the work machine side hitch frame 210, and the frame structure 116 is positioned below the frame structure 211. Then, from that state, the tractor side hitch frame 110 is moved upward. As a result, the tractor side hitch frame 110 supports the work machine side hitch frame 210 from below, and since the connector 160 of the tractor side hitch frame 110 and the receiving side connector (not shown) of the work machine side hitch frame 210 are coupled, the tractor side hitch frame 110 and the work machine side hitch frame 210 are connected.

[0034] As shown in Fig. 4(a), the calibration target part 220 has a marker 222 with a pattern formed by combining a plurality of quadrilaterals. The marker 222 is designed for image recognition at a short distance (for example, about 0.5 m to 1 m). The calibration target part 220 is oriented such that, with the tractor 100 and the working machine 200 connected, the surface on which the marker 222 is provided faces the direction of the tractor 100 (the direction of the camera 140 provided on the tractor 100). Also, when connecting the tractor 100 and the working machine 200, the positions of the camera 140 and the calibration target part 220 are adjusted so that the center of the calibration target part 220 lies on the optical axis of the camera 140.

[0035] Each corner part of each quadrilateral of the marker 222 is recognized as a feature point for image recognition. This feature point serves as a calibration point. The positions of the quadrilaterals in the marker 222 and the positional relationships between the quadrilaterals are known, and the positions and mutual positional relationships between the feature points are also known. Furthermore, the position and orientation (posture) of the marker 222 on the hitch frame 210 (working machine 200) on the working machine side are also known.

[0036] Therefore, by performing calibration using the marker 222, it is possible to obtain the external calibration elements (position and orientation) of the camera 140 that has photographed the marker 222 with respect to the hitch frame 210 (working machine 200) on the working machine side. Note that the principle of calibration is disclosed in Japanese Patent Application Laid-Open No. 2023-65065. In addition to AR markers, coded targets, targets using an appropriate figure as a code, and other known targets can be used as the marker.

[0037] The calibration target part 220 is assumed to be tiltable up and down. For example, as shown in Fig. 4(a), the calibration target part 220 can be tilted slightly upward or, conversely, slightly downward. This up-and-down tilt is to be adjusted according to the position of the camera 140 on the tractor 100 side. Also, for example, in order to reduce the influence of sunlight reflection, the calibration target part 220 may be tilted slightly downward.

[0038] In addition, in the present embodiment, it is also possible to read the ID (work machine ID) of the work machine 200 from the image of the marker 222.

[0039] (Configuration of the control system) FIG. 5(a) is a block diagram showing the control system of the tractor 100. The control system of the tractor 100 includes a control device 120, a camera 140, and a drive system 150.

[0040] The drive system 150 is the power system of the tractor 100 and performs various drives in the tractor 100. As the drive source, an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or hydraulic pressure is used. The drives in the drive system 150 include drives for moving (traveling) the vehicle, drives for steering the vehicle, drives for the hitch frame, drives for the PTO shaft 148, and drives for connecting the PTO shafts.

[0041] FIG. 5(b) schematically shows the hardware configuration of the control device 120. As shown in FIG. 5(b), the control device 120 includes a CPU (Central Processing Unit) 390, a ROM (Read Only Memory) 392, a RAM (Random Access Memory) 394, a storage (here, an SSD (Solid State Drive) or an HDD (Hard Disk Drive)) 396, an input / output interface 397, a display unit 393, an input unit 395, and the like. Each component of the control device 120 is connected to a bus 398. In the control device 120, the functions of each part shown in FIG. 6(a) are realized by the CPU 390 executing a program (including a control program) stored in the ROM 392 or the storage 396. Note that the functions of each part in FIG. 6(a) may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0042] The control device 120 realizes functions as an image acquisition unit 121, a feature quantity detection unit 122, a model discrimination unit 123, a central position estimation unit 124, a marker detection unit 126, an ID determination unit 127, a mounting position calculation unit 128, a movement path calculation unit 129, and a drive control unit 125, as shown in FIG. 6(a), when the CPU 390 executes a program. Note that FIG. 6(a) also shows a work implement DB 130 as a storage unit stored in the storage 396 or the like.

[0043] The image acquisition unit 121 acquires an image captured by the camera 140. The image acquisition unit 121 transmits the acquired image to the feature quantity detection unit 122 and the marker detection unit 126. Note that the image acquisition unit 121 transmits the acquired image to the marker detection unit 126 when the tractor 100 is at a position relatively close to the work implement 200 (when the distance D is equal to or less than the distance D1 shown in FIG. 7(a)). Note that D1 is about 0.5 m to 1 m. In actuality, the distance D means the distance between the lens center of the camera 140 and the -Y side surface of the calibration target unit 220. On the other hand, the image acquisition unit 121 transmits the acquired image to the feature quantity detection unit 122 when the tractor 100 is at a position relatively far from the work implement 200 (when the distance D is greater than the distance D1 shown in FIG. 7(a) and equal to or less than the distance D2). Note that D2 is assumed to be about 5 m to 10 m.

[0044] Returning to FIG. 6(a), the feature quantity detection unit 122 detects the feature quantity of the work implement 200 from the image acquired by the image acquisition unit 121. The feature quantity detection unit 122 transmits the detected feature quantity to the model discrimination unit 123. Note that when the feature quantity detection unit 122 fails to detect the feature quantity, it notifies the drive control unit 125 to that effect. Note that points (coordinates) that are strong against changes in the size and position of an object in the image, changes in the camera viewpoint, illumination, etc. within the image, which can be detected using a method such as ORB (Oriented FAST and Rotated BRIEF), are feature points (keypoints), and the image feature information extracted from the feature points is the feature quantity.

[0045] In the image, the points that can be the above-mentioned feature points are corner points. Feature points are detected mainly using a feature point detection algorithm based on a corner point detection method. Representative feature point detection algorithms include Harris corner detector, Shi-Tomasi detector, SIFT (Scale Invariant Feature Transform), and FAST (Features from Accelerated Segment Test). In the case of ORB described above, FAST is used to detect feature points.

[0046] Regarding the 'Harris corner detector', it is disclosed in 'Harris, C., & Stephens, M. (1988, August). A combined corner and edge detector. In Alvey vision conference (Vol. 15, No. 50, pp. 10-5244). ', etc. Regarding the 'Shi-Tomasi detector', it is disclosed in 'Shi, J. (1994, June). Good features to track. In 1994 Proceedings of IEEE conference on computer vision and pattern recognition (pp. 593-600). IEEE. ', etc. Regarding 'SIFT', it is disclosed in 'Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60, 91-110. ', etc. Furthermore, regarding 'FAST', it is disclosed in 'Rosten, E., & Drummond, T. (2006). Machine learning for high-speed corner detection. In Computer Vision-ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings, Part I 9 (pp. 430-443). Springer Berlin Heidelberg. ', 'FAST algorithm for corner detection, http: / / labs.eecs.tottori-u.ac.jp / sd / Member / oyamada / OpenCV / html / py_tutorials / py_feature2d / py_fast / py_fast.html, accessed date: 2023-10-27', etc.

[0047] The feature quantity (Descriptor) is a local invariant feature descriptor, which is local image feature information extracted from the two-dimensional coordinates that are keypoints in the image. Representative feature quantity extraction algorithms include ORB, SIFT, SURF, BRIEF, FREAK, etc.

[0048] Regarding 'ORB', it is disclosed in, for example, 'ORB (Oriented FAST and Rotated BRIEF), http: / / labs.eecs.tottori-u.ac.jp / sd / Member / oyamada / OpenCV / html / py_tutorials / py_feature2d / py_orb / py_orb.html, accessed date : 2023-10-27' and 'ORB (Oriented FAST and Rotated BRIEF, Oriented FAST and Rotated BRIEF), https: / / whitewell.sakura.ne.jp / OpenCV / py_tutorials / py_feature2d / py_orb / py_orb.html, accessed date : 2023-10-27'. Regarding 'SURF', it is disclosed in, for example, 'Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In Computer Vision-ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings, Part I 9 (pp. 404-417). Springer Berlin Heidelberg.' etc. Regarding 'BRIEF', it is disclosed in, for example, 'Calonder, M., Lepetit, V., Strecha, C., & Fua, P. (2010). Brief: Binary robust independent elementary features. In Computer Vision-ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part IV 11 (pp. 778-792). Springer Berlin Heidelberg.'Furthermore, regarding 'FREAK', it is disclosed in 'Alahi, A., Ortiz, R., & Vandergheynst, P. (2012, June). Freak: Fast retina keypoint. In 2012 IEEE conference on computer vision and pattern recognition (pp. 510-517). Ieee.' and the like.

[0049] FIG. 8 shows an example of the result of detecting feature amounts from an image of the working machine 200.

[0050] Returning to Fig. 6(a), the machine type discriminator 123 refers to the work machine DB 130 and discriminates the machine type of the work machine having a feature amount similar to the feature amount detected by the feature amount detector 122. Here, the work machine DB 130 stores information on a plurality of work machines. Specifically, as shown in Fig. 6(b), in the work machine DB 130, information such as "work machine ID", "machine type of the work machine", and "feature amount" is stored in association with each other. The "work machine ID" is identification information unique to each work machine, and a different character string is set for each work machine. The "machine type of the work machine" is identification information indicating the machine type of each work machine. The "feature amount" is set of data of the feature amounts of each work machine. The feature amount is extracted from an image of each work machine taken in advance or from a 3D model of each work machine taken in advance. If the machine types are the same, it is assumed that the feature amounts are the same. This feature amount includes coordinate values in a three-dimensional coordinate system (work machine coordinate system) set for each work machine as shown in Fig. 7(a). The machine type discriminator 123 compares the feature amount detected by the feature amount detector 122 with the feature amounts of each work machine stored in the work machine DB 130 (calculates the matching relationship) and identifies the machine type of the work machine having the feature amount most similar to the work machine shown in the image. In this case, the matching relationship between the set of feature amounts of each work machine stored in the work machine DB 130 and the feature amounts detected from the image taken by the camera 140 is calculated. When calculating the matching relationship, a matching algorithm such as BFMatcher (Brute Force Matcher) or FLANN (Fast Library for Approximate Nearest Neighbors) can be used. These 'BFMatcher' and 'FLANN' are disclosed in 'Feature Matching, https: / / docs.opencv.org / 4.x / dc / dc3 / tutorial_py_matcher.html, accessed date : 2023-10-27', etc.

[0051] When the model discrimination unit 123 determines that the model of the identified work machine matches the model of a preset target work machine (the work machine to be mounted), it notifies the central position estimation unit 124 to that effect. If they do not match, it notifies the drive control unit 125 to that effect.

[0052] The central position estimation unit 124 reads out the feature amounts of the model (i.e., the model of the preset target work machine) discriminated by the model discrimination unit 123 from the work machine DB 130, and based on the read information and the feature amounts detected by the feature amount detection unit 122, estimates the position of the origin (central position of the work machine) of the work machine coordinate system in Fig. 7(a) in the camera coordinate system of Fig. 7(a) and the orientation of the work machine coordinate system (i.e., the posture of the work machine). In this case, the central position estimation unit 124 estimates the 3D pose of the work machine shown in the image based on the relationship between the feature amounts (2D) detected from the image and the feature amounts (3D) of the target work machine read from the work machine DB 130, and estimates the central position and posture of the work machine from the estimated 3D pose. Note that when making the estimation, for example, it is assumed that the Perspective-n-point (PnP) problem is solved. Further, after the estimation, an additional optimization algorithm (such as RANSAC or Levenberg-Marquardt) is applied to improve the accuracy. The central position estimation unit 124 transmits the information on the estimated central position and posture of the work machine 200 to the movement path calculation unit 129.

[0053] The marker detection unit 126 detects the marker 222 from the image acquired by the image acquisition unit 121. When the marker detection unit 126 can detect the marker 222, it notifies the mounting position calculation unit 128 to that effect. On the other hand, when the marker 222 cannot be detected, the marker detection unit 126 notifies the drive control unit 125 to that effect.

[0054] The ID determination unit 127 identifies the work machine ID of the work machine 200 from the marker 222. Further, the ID determination unit 127 determines whether the identified work machine ID matches the ID of the preset target work machine.

[0055] The mounting position calculation unit 128 extracts the calibration points from among the markers 222, and calculates the coordinates of the mounting position (guidance target) of the working machine 200 (the origin coordinates of the working machine coordinate system within the camera coordinate system) and the orientation of the working machine based on the extracted calibration points of the markers 222. The mounting position calculation unit 128 transmits the coordinates and orientation of the mounting position to the movement path calculation unit 129.

[0056] When the movement path calculation unit 129 receives the information on the central position of the working machine 200 estimated by the central position estimation unit 124 and the orientation of the working machine, it calculates a movement path for approaching the tractor 100 to the working machine 200 based on the received information on the central position of the working machine 200 and the orientation of the working machine. Further, when the movement path calculation unit 129 receives the coordinates of the mounting position and the orientation of the working machine calculated by the mounting position calculation unit 128, it calculates a movement path for moving the tractor 100 to a position where it can be connected to the working machine 200 based on the received coordinates and orientation. The position where the tractor 100 is to be connected to the working machine 200 is a position where the separation distance between the tractor-side hitch frame 110 and the working machine-side hitch frame 210 becomes zero. The movement path calculation unit 129 transmits the information on the calculated movement path to the drive control unit 125.

[0057] The drive control unit 125 performs autonomous control (automatic driving control) related to the movement of the tractor 100 via the drive system 150 based on the notifications from the feature quantity detection unit 122 and the marker detection unit 126 and the movement path calculated by the movement path calculation unit 129. Specifically, the drive control unit 125 performs rotation control and steering control of the drive tires necessary for moving the tractor 100 along the movement path. Further, the drive control unit 125 also performs drive control necessary for connecting the tractor-side hitch frame 110 to the working machine-side hitch frame 210, drive control of the PTO shaft, and drive control for connecting the PTO shaft to the PTO shaft of the working machine 200.

[0058] (Flowchart) Next, the processing of the tractor 100 when connecting the tractor 100 and the working machine 200 will be described in detail with reference to the flowchart of FIG. 9.

[0059] On the premise that the process of FIG. 9 is started, it is assumed that the type and ID of the working machine 200 connected to the tractor 100 are preset (input) in advance. When the process of FIG. 9 is started, first, in step S10, the drive control unit 125 moves the tractor 100 via the drive system 150 to a position where the working machine 200 enters the field of view of the camera 140. As a result, as shown in FIG. 7(a), the distance D between the tractor 100 and the working machine 200 is included in the range where D is greater than D1 and less than or equal to D2. Also, the tractor 100 comes to a position where the rear of the tractor 100 faces the working machine 200. In step S10, the operator may drive the tractor 100 to move the tractor 100 to a position at a predetermined distance D (D1 < D ≦ D2) from the working machine 200.

[0060] Next, in step S12, the image acquisition unit 121 acquires the image captured by the camera 140, and the feature amount detection unit 122 detects the feature amount of the working machine 200 using the image.

[0061] Next, in step S14, the feature amount detection unit 122 determines whether the feature amount can be detected. If the determination in this step S14 is negative, the process returns to step S10. In this case, in step S10, the drive control unit 125 adjusts the position of the tractor 100, and in step S12, the feature amount detection unit 122 detects the feature amount of the working machine 200 again. On the other hand, if the determination in step S14 is affirmative, the process proceeds to step S16.

[0062] Note that the repetition interval for detecting the feature amount shall be adjusted according to the shooting interval of the camera 140. For example, when the camera 140 shoots at an interval of 0.5 seconds, the detection of the feature amount is also performed at an interval of 0.5 seconds.

[0063] When the process proceeds to step S16, the type discrimination unit 123 refers to the working machine DB 130 and selects the type of the working machine having the feature amount closest to the detected feature amount.

[0064] Next, in step S18, the machine type determination unit 123 determines whether the machine type of the selected work machine matches the machine type of the target work machine set in advance. If the determination in this step S18 is negative, since the tractor 100 is about to approach a work machine other than the target work machine, the entire process shown in FIG. 9 ends. On the other hand, if the determination in step S18 is affirmative, the process proceeds to step S20.

[0065] When the process proceeds to step S20, the central position estimation unit 124 estimates the central position and orientation of the work machine 200 shown in the image based on the detected feature amounts. The central position estimation unit 124 transmits the information on the estimated central position and orientation to the movement path calculation unit 129.

[0066] Next, in step S22, the movement path calculation unit 129 calculates a movement path for approaching the tractor 100 to the central position of the work machine 200 based on the information received from the central position estimation unit 124, and the drive control unit 125 moves the tractor 100 along the calculated movement path. Note that the tractor 100 moves so that the camera 140 of the tractor 100 approaches the calibration target portion 220 of the work machine 200 from directly in front.

[0067] Next, in step S24, the marker detection unit 126 detects a marker using the image captured by the camera 140.

[0068] Next, in step S26, the marker detection unit 126 determines whether a marker has been detected. If this determination is negative, the process returns to step S22, and the processes and determinations in steps S22, S24, and S26 are repeated until a marker can be detected. On the other hand, if the determination in step S26 is affirmative, the process proceeds to step S28. Note that the case where the determination in step S26 is affirmative means that, as shown in FIG. 7(b), the tractor 100 approaches the work machine 200 from directly in front and the distance D becomes D1 or less.

[0069] When shifting to step S28, the ID determination unit 127 determines whether the work machine ID obtained from the marker 222 matches the work machine ID of the target work machine set in advance. If this determination is negative, since the tractor 100 is approaching (attempting to attach) a work machine other than the target work machine, the entire process in FIG. 9 is terminated. On the other hand, if the determination in step S28 is affirmative, the process proceeds to step S32.

[0070] When the process proceeds to step S32, the mounting position calculation unit 128 calculates the mounting position and posture of the work machine 200 shown in the image based on the reference points extracted from the marker 222.

[0071] Next, in step S34, the movement path calculation unit 129 calculates a movement path using the mounting position and posture of the work machine 200, and the drive control unit 125 determines control parameters (such as steering angle and speed) for moving along the calculated movement path. Then, the drive control unit 125 controls the drive system 150 using the determined control parameters to move the tractor 100. Note that the method of moving the tractor 100 in step S34 is disclosed in Japanese Patent Application Laid-Open No. 2023-65065.

[0072] Thereafter, in step S36, the drive control unit 125 repeatedly executes steps S32 and S34 until the movement is completed. Then, when the movement is completed, the process proceeds to step S38, and the drive control unit 125 executes the attachment operation of the work implement 200. As described above, the attachment operation aligns the positions of the frame structure 116 of the tractor-side hitch frame 110 and the frame structure 211 of the work implement-side hitch frame 210 in the horizontal direction, and positions the frame structure 116 below the frame structure 211. Then, from that state, the tractor-side hitch frame 110 is moved upward. As a result, the tractor-side hitch frame 110 supports the work implement-side hitch frame 210 from below, and since the connector 160 of the tractor-side hitch frame 110 and the receiving-side connector (not shown) of the work implement-side hitch frame 210 are coupled, the tractor-side hitch frame 110 and the work implement-side hitch frame 210 are connected to each other.

[0073] When the processing up to step S38 is completed as described above, all the processing in FIG. 9 is completed.

[0074] As described in detail above, according to this embodiment, the tractor 100 includes a camera 140 and a control device 120 that controls the driving of the tractor 100 based on the image captured by the camera 140. Then, the control device 120 detects the feature amount of the working machine from the image captured of the working machine (S12), drives the tractor 100 based on the feature amount, and positions it at a position where the marker 222 of the calibration target portion 220 can be captured by the camera 140 (S20, S22). At that position, the marker 222 of the calibration target portion 220 provided on the working machine is captured using the camera 140 (S24), and based on the image, the tractor 100 is driven to a position (mounting position) where it can be connected to the working machine (S32, S34). Thereby, even without using a position detection device such as GNSS, the tractor 100 can be brought close to a position where the marker 222 of the calibration target portion 220 can be detected using the camera 140. Therefore, in an environment where a position detection device such as GNSS cannot be used (such as indoors or a field surrounded by many tall trees), the operation of bringing the tractor 100 close to the working machine can be performed before detecting the marker 222. In this case, since the camera 140 can be used for both detecting the feature amount of the working machine and detecting the marker 222, cost reduction can be achieved.

[0075] Further, in this embodiment, the tractor 100 automatically approaches the working machine 200 and the tractor 100 and the working machine 200 are automatically connected, so there is no need for a person to perform the connection work. Thereby, the occurrence of accidents during work can be suppressed.

[0076] Also, the camera 140 of this embodiment is a monocular camera. Thereby, the cost can be reduced compared to the case of using a stereo camera or the like.

[0077] In addition, in the present embodiment, the control device 120 compares the feature amounts detected from the image of the work implement with the data of the feature amounts of each of the plurality of types of work implements stored in the work implement DB 130, and based on the comparison result, identifies the type (model) of the work implement photographed by the camera 140. Thereby, it is possible to determine whether or not the preset target work implement has been detected.

[0078] In the above embodiment, in the process of FIG. 9, the case where the entire process of FIG. 9 is terminated when the determinations in step S18 and step S28 are negative has been described, but it is not limited to this. For example, when the determinations in step S18 and step S28 are negative, the tractor 100 may be moved to a position where the next work implement enters the field of view of the camera 140, and the process may start from step S10 of FIG. 9. By doing so, the process of FIG. 9 can be repeatedly executed until the tractor 100 is connected to the preset target work implement.

[0079] In the above embodiment, the tractor 100 and the work implement 200 may be provided with a position detection device such as a GNSS sensor. In this case, in an environment where a position detection device such as a GNSS sensor can be used, in step S22 of FIG. 9, the position detection device may be used to bring the tractor 100 closer to the work implement 200.

[0080] In the above embodiment, the case where the tractor 100 is automatically driven to move the tractor 100 to the mounting position of the work implement has been described, but it is not limited to this. For example, the tractor 100 may not have an automatic driving function. In this case, assist information for causing the driver to move along the calculated movement route may be provided. In this case, for example, the movement route may be displayed on a display, and the driver may drive the tractor 100 by looking at the display.

[0081] In the above-described embodiment, the case where the work vehicle is a tractor and the work implement is an agricultural work implement has been described. However, the present invention is not limited to this. The present invention can be applied to work systems other than the agricultural field as long as it is a work vehicle system that requires connecting a work vehicle and a work implement.

[0082] Note that the above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the processing device should have is provided. By executing that program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable storage medium (excluding carrier waves).

[0083] When distributing the program, for example, it is sold in the form of a portable storage medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. Also, the program can be stored in the storage device of a server computer and transferred from the server computer to other computers via a network.

[0084] The computer that executes the program stores, for example, the program recorded on a portable storage medium or the program transferred from a server computer in its own storage device. Then, the computer reads the program from its own storage device and executes the processing according to the program. Note that the computer can also directly read the program from the portable storage medium and execute the processing according to the program. Also, the computer can sequentially execute the processing according to the received program each time the program is transferred from the server computer.

[0085] The above-described embodiment is a preferred example of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the gist of the present invention.

Description of Symbols

[0086] 100 Tractor (Work Vehicle) 120 Control Device 130 Implement DB (Storage Unit) 140 Camera (Imaging Device) 200 Implement 220 Calibration Target Section (Calibration Target) 390 CPU (Computer)

Claims

1. A work vehicle that can be connected to a work implement, comprising: a photographing device; and a control device that controls the driving of the work vehicle based on an image photographed by the photographing device, wherein the control device: detects a feature amount of the work implement from an image of the work implement photographed; drives the work vehicle based on the feature amount to position the work vehicle at a predetermined position where the photographing device can photograph a calibration target provided on the work implement; at the predetermined position, drives the work vehicle to a position where the work vehicle can be connected to the work implement based on an image obtained by photographing the calibration target provided on the work implement using the photographing device. The work vehicle is characterized by the above.

2. The control device: compares the feature amount detected from the image of the work implement with data of the feature amounts of a plurality of types of work implements stored in a storage unit, and identifies the type of the work implement photographed by the photographing device based on the comparison result. The work vehicle according to claim 1, characterized by the above.

3. When the identified type of the work implement is not a predetermined type of work implement, the control device does not perform the process of positioning at the predetermined position. The work vehicle according to claim 2, characterized by the above.

4. The photographing device is a monocular photographing device. The work vehicle according to claim 1, characterized by the above.

5. The control device: reads out identification information of the work implement based on an image obtained by photographing the calibration target using the photographing device; when the identification information of the work implement does not match the predetermined identification information of the work implement, does not perform the process of driving the work vehicle to a position where the work vehicle can be connected to the work implement. The work vehicle according to any one of claims 1 to 4, characterized by the above.

6. A control program for a work vehicle that can be connected to a work implement, the program causing a computer to execute a process of: detecting a feature amount of the work implement from an image of the work implement photographed using a photographing device; driving the work vehicle based on the feature amount to position the work vehicle at a predetermined position where the calibration target provided on the work implement can be photographed using the photographing device; at the predetermined position, driving the work vehicle to a position where the work vehicle can be connected to the work implement based on an image obtained by photographing the calibration target provided on the work implement using the photographing device. The control program for a work vehicle is characterized by the above.

Images