Agricultural machinery, agricultural machinery system, agricultural machinery connection method and program
The agricultural machine system uses a self-propelled moving body with dual calibration targets for precise alignment, addressing the complexity and cost issues in tractor-work machine connections, ensuring safe and cost-effective automatic coupling.
Patent Information
- Application Number
- JP2021175638
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-10-27
- Publication Date
- 2025-07-31
- Estimated Expiration
- 2041-10-27
AI Technical Summary
The connection operation between a tractor and agricultural work machines is complicated and poses a risk of accidents, and existing solutions like expensive measuring devices are not practical for widespread use.
Agricultural machines equipped with a self-propelled moving body and a connecting mechanism, utilizing two types of calibration targets (global and detailed) for precise alignment, where global targets are recognized from a distance and detailed targets provide high accuracy at close range, enabling low-cost alignment using an inexpensive camera.
Enables reliable and stable automatic coupling of the tractor to the work machine with high positional and orientation accuracy, even with an inexpensive camera, reducing the risk of accidents and costs.
Smart Images

Figure 0007716089000002 
Figure 0007716089000003 
Figure 0007716089000004
Abstract
Description
Technical Field
[0001] The present invention relates to a technique for connecting agricultural machines.
Background Art
[0002] There are known work machines that are towed by a tractor or mounted on a tractor to perform various operations. There are many types of work machines. As an example, work machines for performing fertilization, tillage, pesticide spraying, seeding, etc. can be mentioned.
[0003] Depending on the content of agricultural work, a work machine is selected. At this time, a connection operation between the tractor and the work machine is required. This connection operation is complicated, and there is a concern about the occurrence of accidents during the operation. As techniques for dealing with this problem, the technique of Patent Document 1 and the technique of Non-Patent Document 1 are known.
[0004] Patent Document 1 describes a connecting member that makes the connection between a tractor and a work machine easier. Non-Patent Document 1 discloses a technique for automatically connecting a tractor to a work machine using a camera and a target.
Prior Art Documents
Patent Documents
[0005]
Patent Document 1
Non-Patent Documents
[0006]
Non-Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0007] It is conceivable to mount a measuring device on a tractor, measure the positional relationship and the attitude relationship between the tractor and the working machine, automatically drive the tractor based on the measurement result, and connect the tractor and the working machine. Here, considering practicality, it is not a good idea to use expensive measuring devices.
[0008] Against such a background, an object of the present invention is to provide a technique that enables low-cost alignment when connecting a moving body typified by a tractor to an object.
Means for Solving the Problems
[0009] The present invention relates to an agricultural machine connected by a self-propelled moving body, comprising a connecting mechanism with the moving body, and a first type of calibration target and a second type of calibration target arranged at a position facing the moving body when connected to the moving body. The first type of calibration target is for calibration from a relatively long distance, and the second type of calibration target is for calibration from a relatively short distance. Two of the first type of calibration targets are arranged separately on the left and right, and the second type of calibration target is arranged at a position between the two first type of calibration targets.
[0010] In the present invention, the first type of calibration target and the second type of calibration target have a plurality of calibration points. The display constituting the calibration points of the first type of calibration target is larger than the display constituting the calibration points of the second type of calibration target, and the number of calibration points of the second type of calibration target is larger than the number of calibration points of the first type of calibration target. In the present invention, one or both of the first type of calibration target and the second type of calibration target may have an identification surface facing in the direction of the depression angle.
[0011] The present invention relates to the above-described agricultural machine and a camera, the Agricultural machineryAn agricultural machine system that can be connected to and includes a self-propelled mobile body, wherein the camera photographs the calibration target of the second type from the front, captures the calibration target of the second type at the center of the camera's captured image, and sets the field of view such that the calibration target of the first type is out of the shooting range in a state where the calibration target of the second type can be identified.
[0012] Further, the present invention is an agricultural machine connection method for connecting a self-propelled mobile body to an agricultural machine. The mobile body is equipped with a camera for photographing the agricultural machine. The agricultural machine is equipped with a connection mechanism with the mobile body, a calibration target of the first type, and a calibration target of the second type. The calibration target of the first type is for calibration from a relatively long distance, and the calibration target of the second type is for calibration from a relatively short distance. The calibration targets of the first type are arranged in two parts separated left and right, and the calibration target of the second type is arranged at a position between the two calibration targets of the first type. At a distance where the image recognition of the calibration target of the second type by the camera is not possible, the position and orientation of the mobile body with respect to the agricultural machine are detected by image recognition of the calibration target of the first type by the camera. Based on the position and orientation of the mobile body with respect to the agricultural machine, a path for the mobile body to approach the agricultural machine is set, and the mobile body is moved to the agricultural machine according to the set path. By this movement, when the mobile body approaches the agricultural machine, the calibration target of the first type goes out of the shooting range of the camera, and the image recognition of the calibration target of the second type by the camera becomes possible. Thereafter, the distance between the mobile body and the agricultural machine is measured by the calibration target of the second type, and the mobile body is connected to the agricultural machine by adjusting the distance between the mobile body and the agricultural machine.
[0013] In the invention of the above-described agricultural machine connection method, movement control of the moving body along the set path is performed. During the process of the movement control, calibration of the camera is performed using the first type of calibration target and / or the second type of calibration target. Based on the calibration, the actual movement path of the moving body is calculated. The set path and the actual movement path are compared, and based on the result of the comparison, the movement path is set again.
[0014] Also, in the invention of the above-described agricultural machine connection method, there are a first stage where the distance between the moving body and the agricultural machine is relatively far and a second stage where the distance between the moving body and the agricultural machine is relatively close. In the first stage, calibration of the camera is performed using the first type of calibration target. In the second stage, calibration of the camera is performed using the second type of calibration target. The calibration in the second stage has a larger number of calibration points and higher calibration accuracy than the calibration in the first stage.
[0015] The present invention is also a program for causing a computer to execute control for connecting a self-propelled mobile body to an agricultural machine. The mobile body is provided with a camera for photographing the agricultural machine. The agricultural machine is provided with a connection mechanism with the mobile body, a first type of calibration target, and a second type of calibration target. The first type of calibration target is for calibration from a relatively long distance, and the second type of calibration target is for calibration from a relatively short distance. The first type of calibration target is divided into two and arranged on the left and right, and the second type of calibration target is arranged at a position between the two first type of calibration targets. When the computer cannot recognize the image of the second type of calibration target by the camera at a distance where the image recognition of the first type of calibration target by the camera is possible, detecting the position and orientation of the mobile body with respect to the agricultural machine by image recognition of the first type of calibration target by the camera, setting a path for the mobile body to approach the agricultural machine based on the position and orientation of the mobile body with respect to the agricultural machine, controlling the movement of the mobile body to the agricultural machine according to the set path, and when the mobile body approaches the agricultural machine by the movement control, the first type of calibration target moves out of the shooting range of the camera, and the image recognition of the second type of calibration target by the camera becomes possible, and then measuring the distance between the mobile body and the agricultural machine by the second type of calibration target, and connecting the mobile body to the agricultural machine by adjusting the distance between the mobile body and the agricultural machine.
Advantages of the Invention
[0016] According to the present invention, a technique for enabling alignment of a mobile body and an object at low cost can be obtained.
Brief Description of the Drawings
[0017]
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Mode for Carrying Out the Invention
[0018] 1. Overview FIG. 1 shows an overview of the embodiment. FIG. 1 shows a tractor 100 and a working machine 200. In this example, the working machine 200 is, for example, a working machine for fertilizing. Examples of the types of working machines include those for fertilizing, tilling, pesticide spraying, sowing, etc. There are also multiple types of tilling, and there may be cases where working machines are prepared for each use. Here, there is one working machine, but there may be multiple working machines, and one of them may be connected to the tractor.
[0019] (Connection Structure) The tractor 100 and the working machine 200 are connected by a hitch frame described below. Note that it is also possible for the working machine 200 not to have wheels and to be mounted on the rear of the tractor 100 in a floating form in the air. There is such a form of working machine for fertilizing.
[0020] (Tractor Side Hitch Frame) The tractor 100 is provided with a tractor side hitch frame 110 at its rear. The working machine 200 is provided with a working machine side hitch frame 210 for connecting to the tractor side hitch frame 110.
[0021] FIG. 2 is a perspective view (A) and (B) of the tractor-side hitch frame 110. FIG. 2(A) is a perspective view seen from the attachment side to the tractor 100, and FIG. 2(B) is a perspective view seen from the side facing the implement-side hitch frame 201.
[0022] Hereinafter, the left and right are defined based on the case of viewing the tractor 100 from the rear. Also, the direction in which the tractor 100 moves forward is defined as the front.
[0023] The tractor-side hitch frame 110 includes an inverted V-shaped frame structure 116 that forms the hypotenuse of a triangle. At the lower part of the frame structure 116, there are provided a lower right support portion 111 and a lower left support portion 112 that extend forward. The lower right support portion 111 is provided with a lower right support pin 111a at its tip, and the lower left support portion 112 is provided with a lower right support pin 112a at its tip.
[0024] The lower right support pin 111a is connected to a lower right link 141 (see FIG. 3) arranged at the rear of the tractor 100. As shown in FIG. 3, at the rear of the tractor 100, a lower right link 141 that extends rearward is provided, and at the tip of the lower right link 141, a support hole 141a serving as a lower right hitch point is provided. By inserting and engaging the lower right support pin 111a shown in FIG. 2(A) into this support hole 141a, the lower right support portion 111 is connected to the lower right link 141 on the tractor 100 side.
[0025] Similarly, at the rear of the tractor 100, a lower left link 142 that extends rearward is provided, and at the tip of the lower left link 142, a support hole 142a serving as a lower left hitch point is provided. By inserting and engaging the lower left support pin 112a shown in FIG. 2(A) into this support hole 142a, the lower left support portion 112 is connected to the lower left link 142 on the tractor 100 side.
[0026] As described above, the right lower support portion 111 in Figure 2 is connected to the right lower link 141 on the tractor 100 side in Figure 3, and the left lower support portion 112 in Figure 2 is connected to the left lower link 142 on the tractor 100 side in Figure 3, thereby attaching the tractor side hitch frame 110 to the rear of the tractor 100.
[0027] As shown in FIG. 2(A), an upper support portion 190 is disposed on the tractor-side hitch frame 110. The upper support portion 190 is provided with a long hole and several round holes. The tip of the upper support arm 143 on the tractor 100 side shown in FIG. 3 is rotatably engaged with one of these holes via a horizontal rod (not shown). The upper support portion 190 is provided with multiple holes so that it can accommodate tractors of different sizes and models. The holes to be used in the upper support portion 190 are selected depending on the model and size of the tractor.
[0028] In this way, the tractor-side hitch frame 110 is attached to the rear of the tractor 100 by the right lower link 141, left lower link 142, and upper support arm 143 on the tractor 100 side. The upper support arm 143 is power-operated and can be extended and retracted, and the forward and backward tilt of the tractor-side hitch frame 110 can be adjusted by adjusting the degree of extension and retraction.
[0029] The tip of the right lower link 141 of the tractor 100 can be lifted by the right lift rod 144, and the tip of the left lower link 142 can be lifted by the left lift rod 145. Here, when the right lift rod drive unit 146 rotates, the right lift rod 144 moves back and forth in its axial direction, causing the tip of the right lower link 141 to move up and down. When the left lift rod drive unit 147 rotates, the left lift rod 145 moves back and forth in its axial direction, causing the tip of the left lower link 142 to move up and down. This up and down movement causes the tractor-side hitch frame 110 to move up and down relative to the tractor 100.
[0030] When the tractor-side hitch frame 110 moves up and down relative to the tractor 100, the tractor-side hitch frame 110 tilts forward and backward. In this case, the tractor-side hitch frame 110 is adjusted to be vertical by extending and retracting the upper support arm 143. The position and posture of the tractor-side hitch frame 110 on the tractor 100 are predetermined and known.
[0031] A PTO (Power Take Off) shaft 148 protrudes rearward from the rear of the tractor 100. The PTO shaft 148 is connected to a spline shaft 117 of the tractor-side hitch frame 110 via an expandable joint (not shown).
[0032] The spline shaft 117 has teeth formed on its outer periphery that extend in the axial direction, and is capable of being driven to move back and forth in the axial direction. This mechanism will be explained below. First, a linear bearing holding plate 170 is fixed to the tractor-side hitch frame 110. A linear bearing 171 is fixed to the linear bearing holding plate 170.
[0033] The linear bearing 171 is a sliding bearing, and holds the sliding shaft 172 in a state where it can slide in the axial direction. The sliding shaft 172 has a pin 172a that protrudes in a direction perpendicular to the axis, and this pin 172a is engaged with a fork assembly (link) 113. The fork assembly (link) 113 is driven to rotate by an electric cylinder 114 via a drive arm 115. The rotation of this fork assembly (link) 113 moves the sliding shaft 172 back and forth.
[0034] A bearing holding plate 173, on which the spline shaft 117 is rotatably supported, is fixed to the rear end (the end on the side of the work machine 200) of the sliding shaft 172. As the sliding shaft 172 moves back and forth, the spline shaft 111 moves back and forth (axially) together with the bearing holding plate 173.
[0035] After the tractor-side hitch frame 110 and the work implement-side hitch frame 210 are connected, the electric cylinder 114 is operated to slide the sliding shaft 172 rearward (toward the work implement). This causes the bearing retaining plate 173 holding the spline shaft 117 to move rearward (toward the work implement 200), and the spline shaft 117 connects with the spline sleeve shaft 212 (see FIG. 4) of the spline sleeve structure on the work implement 200 side. In this way, the PTO shaft 148 of the tractor 100 and the PIC shaft (not shown) of the work implement 200 are connected. The PIC shaft is called the power input connect shaft and is the shaft on the work implement side that receives driving force from the tractor 100.
[0036] The tractor-side hitch frame 110 also has a hook 119 that is driven by an electric cylinder 118. When the hook 119 is rotated by the electric cylinder 118, the hook 119 is caught in an opening 218 provided in the work implement-side hitch frame 210 shown in Figure 4. This locks the tractor-side hitch frame 110 and the work implement-side hitch frame 210 in a connected state.
[0037] A connector 160 is disposed on the top of the tractor-side hitch frame 110. The connector 160 is coupled to a receiving connector on the work implement-side hitch frame 210. The connector 160 connects the hydraulic system and various electrical wiring between the tractor 100 and the work implement 200.
[0038] Reference numerals 161 and 162 denote hydraulic transmission pipes connecting the connector 160 and the tractor 100. Reference numeral 163 denotes an electrical wiring cable connecting the connector 160 and the tractor 100.
[0039] (Hitch frame on work equipment side) Next, we will explain the work implement side hitch frame 210 that is connected to the tractor side hitch frame 110. Fig. 4(A) is a perspective view of the work implement side hitch frame 210, looking at the side facing the tractor 100. Fig. 4(B) is a perspective view of the work implement 200 side.
[0040] The hitch frame 210 on the work machine side has a triangular frame structure 211. This triangular frame structure 211 has a U-shaped structure that is recessed inward, and the frame structure 116 of the tractor-side hitch frame 110 is configured to fit into it from below inside.
[0041] With the horizontal positions of the frame structure 116 and the frame structure 211 aligned, with the frame structure 211 positioned above and the frame structure 116 positioned below, by moving the tractor-side hitch frame 110 upward, the frame structure 116 of the tractor-side hitch frame 110 fits into the inside of the frame structure 211 of the work machine-side hitch frame 210 from below, and the tractor-side hitch frame 110 and the work machine-side hitch frame 210 are coupled.
[0042] In this state, the tractor hitch frame 110 and the work machine-side hitch frame 210 cannot move apart front and back, and the tractor hitch frame 110 supports the work machine-side hitch frame 210 from below. Also, in this state, the connector 160 of the tractor-side hitch frame 110 is coupled to the receiving-side connector housed inside the portion numbered 219 of the work machine-side hitch frame 210.
[0043] Below the work machine-side hitch frame 210, a spline slip shaft 212 supported by a bearing structure is arranged. The spline shaft 117 of the tractor-side hitch frame 110 is connected to the spline slip shaft 212. Also, the side of the spline slip shaft 212 on the work machine 200 side is connected to a PIC shaft (not shown) on the work machine 200 side.
[0044] The inner edge portion and the tooth end portion of the part where the spline shaft 117 of the spline slip shaft 212 fits are tapered, and are designed to facilitate the fitting of the spline shaft 117. For the same purpose, the end of the spline shaft 117 is also tapered. The fitting of the spline shaft 117 into the spline slip shaft 212 is possible even when the spline shaft is rotated (of course, it is also possible without rotation).
[0045] As shown in FIG. 4(B), on the side of the working machine 200 of the hitch frame 210 on the working machine side, an upper fixing member 213 and lower fixing members 215, 215 for fixing the hitch frame 210 on the working machine side to the working machine 200 (see FIG. 1) are arranged. The position and orientation of the hitch frame 210 on the working machine side in the working machine 200 are predetermined and known.
[0046] Reference numerals 231 and 232 are connection ports for transmitting the hydraulic pressure from the receiving-side connector arranged inside the portion of reference numeral 219. A hydraulic cable (not shown) of the working machine 200 is connected to this connection port. Reference numeral 233 is a connector for the electrical wiring connected to the above receiving-side connector. Wiring from the working machine 200 is connected to this connector 233.
[0047] (Calibration target) As shown in FIG. 4, a calibration target portion 220 is attached to the upper part of the hitch frame 210 on the working machine side. The calibration target portion 220 is composed of global targets 221a and 221b arranged two apart from each other left and right, and a detailed target 222 (see FIG. 6 for details) arranged between the global targets 221a and 221b.
[0048] The global targets 221a, 221b and the detailed target 222 are adjusted in orientation so that their surfaces face the direction of the tractor 100 (the direction of the camera 140) in a state where the tractor 100 and the working machine 200 are connected. In this example, in a state where the tractor 100 and the working machine 200 are connected, the global targets 221a, 221b and the detailed target 222 are set to face the tractor 100 directly.
[0049] The global targets 221a, 221b are calibration targets assumed for image recognition at a relatively long distance (about 2 m to 5 m or more in this example). The detailed target 222 is a calibration target assumed for image recognition at a relatively short distance (less than about 2 m to 5 m in this example).
[0050] The global targets 221a and 221b have a display that combines multiple rectangular designs. The corners of each rectangle are image-recognized as feature points. These feature points become orientation points. The positions of each rectangle in the global targets 221a and 221b, as well as the relative positions of the rectangles, are known, and the positions of the feature points and their relative positions are also known.
[0051] The positional relationship between the global targets 221a and 221b is also known. Furthermore, the positions and orientations (postures) of the global targets 221a and 221b on the work machine side hitch frame 210 (work machine 200) are also known.
[0052] Therefore, by performing orientation using the global targets 221a and 221b, it is possible to determine the exterior orientation parameters (position and attitude) of the camera 140 that photographed the global targets 221a and 221b with respect to the work implement side hitch frame 210 (work implement 200). The principle of orientation will be described later.
[0053] The designs of the global targets 221a and 221b are set to be relatively large so that they can be recognized from a distance. However, because the designs are large, the density of the orientation points decreases, and the number of orientation points is also limited.
[0054] The detailed target 222 has a more detailed pattern and a larger number of feature points than the global targets 221a and 221b. This allows for a larger number of orientation points to be obtained, enabling orientation with higher accuracy. The position and orientation of the detailed target 222 relative to the work implement hitch frame 210 and the work implement 200 are also known.
[0055] In this example, the detailed target 222 is composed of a large number of rectangular AR markers arranged vertically and horizontally in a matrix. Each AR marker is composed of a combination of a large number of rectangular patterns, and the corners of each rectangular pattern are extracted from the image as feature points, which become orientation points. The positions of the corners of the patterns that become the orientation points for each AR marker are known, and the relative positions of the AR markers arranged vertically and horizontally are also known. Therefore, a large number of orientation points can be obtained from the captured image of the detailed target 222.
[0056] The AR marker is just one example, and coded targets, targets using appropriate graphics as codes, and other known targets can also be used. The important thing is that they can be used as location targets.
[0057] The target display surfaces of the global targets 221a, 221b and the detailed target 222 are flat, and designs for image recognition of the orientation points are displayed on this flat surface.
[0058] The location target unit 220 can be tilted up and down. Figure 4 shows the location target unit 220 tilted slightly upward. This vertical tilt is adjusted taking into account the vertical positional relationship of the camera 140 on the tractor 100 side.
[0059] For example, by arranging the orientation target unit 220 so that it is tilted slightly downward, the effect of sunlight reflection can be reduced. If sunlight is reflected from the orientation target unit 220, it may be difficult to recognize the image of the orientation pattern. As a countermeasure in this case, the orientation target unit 220 is arranged so that it is tilted slightly downward (in the direction of the depression angle). Since sunlight does not hit the orientation target unit 220 from the depression angle, by arranging the orientation target unit 220 so that it is tilted slightly downward, it is possible to avoid the problem of sunlight being reflected from the orientation target unit 220 and appearing in the image captured by the camera 140.
[0060] The tilt angle is approximately 10° to 45°. If the tilt angle is known, orientation can be performed without any problems. A configuration in which one of the global target and the detailed target is arranged facing downward is also possible. Furthermore, the global targets 221a, 221b and the detailed target 222 shown in FIG. 4 may be arranged vertically, and in addition, the global target and / or the detailed target may be arranged tilted downward.
[0061] (Principle of orientation) Orientation will be explained below. Orientation is the process of finding the exterior orientation parameters (position and orientation) of the camera using orientation points. Here, we will explain single photo orientation, which performs orientation from a single photo. Figure 5 is a diagram illustrating the principle of the intersection method, which is a basic method for finding the position and orientation of the camera in single photo orientation.
[0062] The basic principle is that if there are three or more points (calibration points) with known three-dimensional coordinates on a single photograph, the position (three-dimensional position) and attitude (the direction in which it is facing in the three-dimensional coordinate system) of the camera that took the photograph can be determined.
[0063] This principle will be explained below with reference to Figure 5. Point O is the viewpoint of photography (the projection origin of the camera used). P1, P2, and P3 are control points whose three-dimensional positions (three-dimensional coordinates) are known. p1, p2, and p3 are the positions (photo coordinates) of points P1, P2, and P3 on the photographic screen obtained by photography.
[0064] In this case, a line is set geometrically connecting points p1 and P1, a line connecting points p2 and P2, and a line connecting points p3 and P3. The position of the intersection of these three lines becomes the coordinates of point O. Since the positions of points P1, P2, and P3 are known, the coordinates of point O in the coordinate system describing points P1, P2, and P3 can be determined.
[0065] The straight line connecting point O and the center of the shooting screen is the optical axis of the camera. In this way, the position and orientation of the camera in the coordinate system describing points P1, P2, and P3 are determined.
[0066] The above is the basic principle. Hereinafter, the actual calculation method will be described. In this embodiment, the DLT method is used to obtain the position and orientation of the camera. The DLT method approximates the relationship between the photographic coordinates and the three-dimensional coordinates of the subject by a third-order projective transformation formula. The calculation formula is shown in Equation (1) below.
[0067]
Equation
[0068] In the DLT method, if there are nine or more points with known positions in a single photographic screen, the position and orientation of the camera in the coordinate system describing their positions can be obtained. In the above case, L1 to L 11 become unknown parameters related to the position and orientation of the camera.
[0069] In the above manner, the position and orientation of the camera 140 with respect to the calibration target unit 220 are obtained. Since the position and orientation of the camera 140 in the tractor 100 and the position and orientation of the calibration target unit 220 in the working machine 200 are known, as a result, the positional relationship and the orientation relationship between the tractor 100 and the working machine 200 are obtained by the above calibration. For example, the separation distance between the tractor 100 and the working machine 200 and the orientation relationship between the tractor 100 and the working machine 200 are obtained. In addition, the positional and orientation relationships between the tractor-side hitch frame 110 and the working machine-side hitch frame 210 are obtained.
[0070] Here, the larger the number of calibration points, and the wider the range (up to a range with a viewing angle opening of 90°) the calibration points are in rather than a narrow range as viewed from the camera, the higher the calculation accuracy of the position and orientation of the camera 140 will be.
[0071] In this example, by arranging the large-area targets 221a and 221b, which are enlarged so as to enable image recognition even from a distance, apart from each other left and right, a plurality of calibration points are distributed apart from each other left and right, ensuring the accuracy of calibration at a long distance. On the other hand, the detailed target 222 used at a short distance ensures the accuracy of calibration by increasing the number of calibration points.
[0072] (Appearance of the target) FIG. 6 shows the transition of the appearance of the calibration target portion 220 shown in the captured image of the camera 140. FIG. 6(A) shows the case where the calibration target portion 220 is photographed from a distance that fits within the captured screen, and FIG. 6(B) shows the case where the calibration target portion 220 protrudes from the captured screen as it gets closer from the distance in FIG. 6(A).
[0073] When the distance is far, the display of the detailed target 222 is difficult to recognize by image recognition, and calibration using the detailed target 222 is difficult or has a large error. The error increases because the display is blurred, so the positional accuracy of the feature points in the screen decreases, and the mis-extraction or non-extraction of the feature points increases. This becomes prominent when an inexpensive camera 140 is used.
[0074] In this case, calibration is performed using the large-area targets 221a and 221b with a large display. At this time, since the large-area targets 221a and 221b are arranged apart from each other left and right, the calibration accuracy in the left-right direction can be increased even with a small number of calibration points. Therefore, information on the position and orientation of the working machine 200 of the tractor 100 can be obtained with high accuracy.
[0075] In the case of FIG. 6(B), since the camera 140 approaches the calibration target portion 220, the large-area targets 221a and 221b are out of the shooting range (angle of view) of the camera 140, and the detailed target 222 mainly occupies the screen.
[0076] Conversely, when approaching, the relationship between the sizes of the patterns of the large-area targets 221a, 221b and the detailed target 222 and the optical characteristics (magnification and focal length) of the camera 140 is set so that the large-area targets 221a and 221b are out of the way and the detailed target 222 is mainly shown.
[0077] 6(B), the detailed target 222 with many control points is used to determine the relationship between the positions and attitudes of the tractor 100 and the work implement 200. This makes it possible to determine with high accuracy the position and attitude (mainly position (distance) at this stage) of the tractor 100 relative to the work implement 200 at the final stage when the tractor-side hitch frame 110 comes into contact with the work implement-side hitch frame 210.
[0078] In particular, the detailed target 220 can obtain a large number of control points distributed vertically and horizontally, allowing for precise measurement of the precise distance required in the final stage (the distance between the tractor side hitch frame 110 and the implement side hitch frame 210).
[0079] (Control system configuration) 7 is a control block diagram of the tractor 100. The tractor 100 is equipped with a control computer 120, a GPS device 130, a camera 140, and a drive system 150. The control computer 120 is equipped with a CPU, a storage device, and various interface devices.
[0080] The control computer 120 configures an image data acquisition unit 121, a target recognition unit 122, a control point extraction unit 123, a guidance target detection unit 124, a location processing unit 125, a movement path setting unit 126, a vehicle control unit 127, a guidance determination unit 128, and a location-based movement path calculation unit 129. These functional units may be configured as software or may be configured as dedicated hardware. It is also possible to configure some or all of the above functional units using FPGA or the like.
[0081] The image data acquisition unit 121 acquires image data of an image captured by the camera 140. The target recognition unit 122 detects global targets 221a, 221b and a detailed target 222 from the image captured by the camera 140. The orientation point extraction unit 123 extracts orientation points from the global targets 221a, 221b and the detailed target 222 that have been image-recognized.
[0082] The guidance target detection unit 124 detects a portion (guidance target) that is a target for guiding the tractor 100. In this example, the central position between the global targets 221a and 221b that are spaced apart on the left and right is set as the guidance target. Note that the central position (guidance target) between the global targets 221a and 221b is set to coincide with the center position of the detailed target 222.
[0083] The orientation processing unit 125 orients the camera 140 using the orientation target unit 220 (global targets 221a, 221b and detailed target 222). This orientation determines the position and orientation of the camera 140 relative to the orientation targets. The position and orientation of the camera 140 on the tractor 100 are known, and the position and orientation of the orientation target 220 relative to the work implement 200 are also known, so determining the position and orientation of the camera 140 relative to the orientation target 220 determines the position and orientation of the tractor 100 relative to the work implement 200. In addition, the relationship between the positions and orientations of the tractor-side hitch frame 110 and the work implement-side hitch frame 210 is determined.
[0084] Based on the position and attitude of the tractor 100 relative to the work implement 200 determined by the above orientation, the movement path setting unit 126 sets a movement path of the tractor 100 relative to the work implement 200 required to couple the tractor 100 to the work implement 200. In other words, it sets a planned movement path of the tractor 100 relative to the work implement 200 required to couple the tractor side hitch frame 110 to the work implement side hitch frame 210.
[0085] An example of the planned movement path is shown in Figure 8. In the case of Figure 8, a path is set that involves two turns so that the back of the tractor 100 faces the work implement 200 directly (the difference in direction is 0°) at a position P1, a distance L1 before the final target position.
[0086] P1 is set as the position where the global targets 221a and 221b are out of the field of view of the camera 140 (not necessarily completely out of the field of view), mainly the detailed target 222 appears in the captured image, and the center of the captured image coincides (or may be approximately coincident) with the center (guidance target) of the detailed target 222. L1 is set to about 50 cm to 1 m. The target position is the position where the tractor-side hitch frame 110 and the implement-side hitch frame 210 are connected (the position where the separation distance between the tractor-side hitch frame 110 and the implement-side hitch frame 210 becomes 0).
[0087] The planned movement path in FIG. 8 is set as follows. First, the starting point PA is the position where the calibration of the camera 140 using the global targets 221a and 221b becomes possible. Then, based on the positional relationship between the starting point PA obtained as a result of the calibration of the camera 140 and the target position, the relationship between the orientations of the tractor 100 and the implement 200, and the direction of the front of the implement 200, the front position P1 is obtained. The direction of the front of the implement 200 is obtained from the direction of the surface of the calibration target portion 220 with respect to the implement 200.
[0088] The front position P1 is set as the position where the camera 140 can recognize the detailed target 222 for calibration and the tractor-side hitch frame 110 faces the implement-side hitch frame 220 (the orientations in the horizontal direction are the same).
[0089] Next, the midpoint C1 between PA and P1 is obtained, and the path of the tractor reaching P1 by two left and right turns with the same turning radius is set. In this way, the planned movement path in FIG. 8 is set.
[0090] As the planned movement path, it is also possible to have a form where the tractor makes one turn or three or more turns. For example, a path where the tractor-side hitch frame 110 faces the implement-side hitch frame 210 at the position of P1 while repeatedly making small turns left and right is possible.
[0091] The vehicle control unit 127 performs autonomous control (control of automatic driving) related to the movement of the tractor 100. Specifically, it performs rotation control and steering control of the drive tires required to move the tractor 100 along the movement path from PA to P1 shown in Fig. 8, and rotation control and steering control of the drive tires required to move the tractor 100 from P1 to the target position.
[0092] The vehicle control unit 127 also performs the drive control required to connect the tractor side hitch frame 110 to the work implement side hitch frame 210, drive control of the PTO shaft, and drive control for connecting the PTO shaft to the PTO shaft of the work implement 200.
[0093] The guidance determination unit 128 determines whether the tractor 100 is moving along the movement path shown in Figure 8, and whether the tractor is being properly guided as it approaches the work implement 200 from P1.
[0094] The orientation-based movement path calculation unit 129 calculates the actual movement path of the tractor 100 based on the results of orientation of the camera 140 using the global targets 221 a and 221 b and / or the detailed target 222 .
[0095] For example, suppose the above orientation is performed every second while the tractor 100 is moving. In this case, the position of the camera 140 relative to the work implement 200 is obtained every second. By tracking the plot of this position, the actual movement path of the tractor 100 relative to the work implement 200 can be obtained.
[0096] For example, by drawing a line connecting the plotted points on the diagram of Fig. 8, it is possible to compare the set movement path with the actual movement path of the tractor 100. For example, by making this comparison, it is possible to determine whether the guidance is being performed appropriately.
[0097] The GPS device 130 performs positioning using navigation signals from navigation satellites. Although not shown in the figure, in addition to the GPS device 130, a direction sensor and a gyro sensor are also arranged on the tractor 100, and not only the position of the tractor 100 but also its attitude is measured. The relationships between the positions and attitudes of the GPS device 130, the direction sensor, and the gyro sensor in the tractor 100 are pre-measured and are known data.
[0098] The camera 140 is arranged behind the tractor 100 and is set to photograph the rear of the tractor 100. In this example, the camera 140 is arranged directly behind on the central axis of the tractor 100. The position and attitude of the camera 140 on the tractor 100 are pre-measured and are known data. Also, the positions of the camera 140 and the calibration target unit 220 are adjusted so that the center of the calibration target unit 220 (the center of the detailed target 222: the guiding target) comes on the optical axis of the camera 140 during connection.
[0099] The camera 140 is a digital still camera and continuously shoots still videos. The shooting interval is set to 0.25 seconds to 1 second. It is also possible to shoot a video and use the frame images constituting the video as still images.
[0100] The camera 140 may be an industrial camera or an inexpensive one such as a web camera, rather than a digital still camera. Inexpensive cameras have a loose focus adjustment (some are unadjusted) so that they can shoot over a wide distance range, and the resolution of distant images is low. However, since calibration at a long distance is performed with a global target and calibration at a short distance is performed with a detailed target, even inexpensive cameras can be used.
[0101] Note that the camera 140 is calibrated in advance to obtain internal calibration elements (correction parameters for correcting optical system distortion, etc.).
[0102] The drive system 150 is a power system of the tractor 100, and performs various driving operations in the tractor 100. The drive source may be an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or hydraulics.
[0103] The drive in the drive system 150 includes drive for moving (driving) the vehicle, drive for steering the vehicle, drive for the hitch frame, drive for the PTO shaft, and drive for connecting the PTO shaft.
[0104] (Example of processing) Fig. 9 is a flowchart showing an example of processing performed in control computer 120. A program for executing the processing in Fig. 9 is stored in a semiconductor storage device or hard disk drive provided in control computer 120, and is read and executed by the CPU of control computer 120. The program may also be stored in an appropriate storage medium.
[0105] When the process starts, first, the camera 140 starts taking pictures (step S101). Next, the tractor 100 is guided to the vicinity of the target work implement by using GPS or by manual driving by the driver (step S102). This guidance is performed so that the tractor 100 is positioned within a distance of about 5 to 10 meters from the target work implement, and so that the back of the tractor 100 faces the work implement 200 as much as possible. The position of the tractor 100 is defined as the position of the center of gravity of the tractor 100 or the center of the vehicle. The same applies to the position of the work implement 200.
[0106] For example, the position information of the work machine 200 when it was last used is recorded, and the above-mentioned guidance using the GPS is performed based on this information.
[0107] In this guiding process, it is determined whether the camera 140 can recognize the global targets 221a and 221b through image recognition (step S103). In the case of this example, when approaching a distance of 10 m, the settings of the camera 140 and the patterns of the global targets 221a and 221b, or the patterns of the global targets 221a and 221b are set according to the performance of the camera 140 so that the global targets 221a and 221b can be recognized through image recognition. This image recognition is performed by the target recognition unit 122.
[0108] If the global targets 221a and 221b can be recognized (captured) through image recognition, the tractor 100 is temporarily stopped at that position, and calibration using the global targets 221a and 221b is started (step S104). This process is performed by the calibration processing unit 125. By performing this calibration, the relationship between the position and posture (orientation) of the tractor 100 with respect to the working machine 200 at that time is determined. This calibration is continuously repeated hereafter.
[0109] The interval of repeating the calibration is adjusted according to the shooting interval of the camera 140. For example, when the camera 140 shoots at an interval of 0.5 seconds, the calibration is also performed at an interval of 0.5 seconds.
[0110] Once the relationship between the position and posture (orientation) of the tractor 100 and the working machine 200 at the temporarily stopped position is determined, a movement path from the starting point PA to the forward target P1 is set by the method shown in FIG. 8 (step S105). Here, the movement path in FIG. 8 is set with the temporarily stopped position as the starting point PA. This process is performed by the movement path setting unit 126.
[0111] Once the movement path is set, the tractor is moved along the movement path (step S106). In this example, the vehicle control unit 127 performs automatic driving along the set movement path. That is, the steering control and the rotation control of the drive wheels are performed so that the tractor 100 autonomously moves on the set movement path.
[0112] Instead of automatic driving, it is also possible to provide the driver with information to assist him / her in traveling along the travel route, and then manually drive the tractor 100 along the set travel route. In this case, the route is displayed on a display, for example, and the driver drives the tractor 100 while looking at it.
[0113] After the guidance is started, the location in step S104 is continuously and repeatedly performed. Here, it is determined whether the guidance in step S106 is appropriate or not (step S107). This process is performed by the guidance determination unit 128.
[0114] The determination in step S107 will be described below. During the guidance process, the orientation of the camera 140 is continuously and repeatedly performed, and the position and attitude of the camera 140 are calculated every moment. This information is used to calculate the actual movement path of the tractor 100. This calculation of the actual movement path is performed by the movement path calculation unit 129 based on the orientation.
[0115] By comparing the calculated actual movement route with the planned movement route set in step S105, it is determined whether the set route is being accurately traced. Note that, in calculating the actual movement route of the tractor 100, GPS information and / or attitude information based on a direction sensor or a gyro sensor may be used.
[0116] For example, the travel path of the tractor 100 obtained from the above-mentioned orientation results is created as an actually measured travel path. Then, the actually measured travel path is compared with the travel path set in step S105 (planned travel path). If there is no deviation between the two, the guidance at that time is determined to be appropriate. If there is an unacceptable deviation between the two, the guidance at that time is determined to be inappropriate.
[0117] If it is determined that the guidance is inappropriate, the processes from step S105 onward are repeated. In this case, a new travel route is set in step S105. The new travel route is set with the position of the tractor 100 at the time of the above determination as the starting point PA and the position of point P1 in Figure 8 as the end point.
[0118] The driving wheels (drive wheels) of a tractor are large and not suitable for precise control, and various factors such as road surface conditions not being ideal can sometimes prevent the tractor 100 from accurately tracing the set movement path. In such cases, a new movement path is set using the method described above. This gradually updates the movement path to the target position (in this case, P1 in Figure 8), improving the accuracy of reaching the target position.
[0119] If the guidance is performed appropriately, it is determined whether or not the detailed target 222 has been recognized (captured) in the image captured by the camera 140 (step S108). In this example, the camera 140 and the design of the detailed target 222 are set so that the detailed target 222 can be image-recognized when the target approaches within a distance of 3 m. This image recognition is performed by the target recognition unit 122.
[0120] If the detailed target 222 cannot be captured, the processes from step S106 onwards are repeated. If the detailed target 222 can be captured, orientation of the camera 140 using the detailed target 222 is started (step S109). This process is performed by the orientation processing unit 125. The orientation performed in step S109 uses more orientation points than the orientation performed in step S104, and is therefore performed with higher accuracy. This orientation is then continuously and repeatedly performed.
[0121] At the stage of step S109, the measurement of the position and orientation of the work implement 200 with respect to the tractor 100 switches from being based on calibration using the global targets 221a, 221 to being based on calibration using the detailed target 222. Note that it is also possible to perform calibration using the detailed target 222 after continuing the calibration using the global targets 221a, 221b.
[0122] Next, it is determined again whether the guidance is appropriate (step S110). The method of determination is the same as in step S107. However, since the position and orientation information used is more accurate than in the case of step S107, the determination conditions become stricter. For example, assume that the allowable error in step S107 is 3 cm. In this case, the allowable error in step S110 is, for example, 5 mm.
[0123] If it is determined in the determination of step S110 that the guidance is not appropriate, the process returns to the stage before step S105, and a new travel route is set again. The new travel route is set with the position of the tractor 100 at the time when the determination of step S110 is made as the starting point PA and the position of point P1 in FIG. 8 as the end point.
[0124] In the determination of step S110, the detailed target 222 with more calibration points than the global targets 221a, 221b is used. For this reason, the calibration accuracy is higher than in the case of step S104. As a result, the closer the tractor 100 approaches the work implement 200, the higher the calibration accuracy, that is, the measurement accuracy of the position and orientation of the tractor 100 with respect to the work implement 200. And the accuracy of reaching point P1 in FIG. 8 can be improved.
[0125] Also, when a new travel route is set again, the accuracy of the setting of P1 also increases. In this respect as well, the accuracy of the connection work between the final tractor-side hitch frame 110 and the work implement-side hitch frame 210 is improved.
[0126] In step S110, when it is determined that the guidance is appropriate, it is determined whether it is the forward target position P1, that is, whether the position of the tractor 100 is P1 (step S111). The forward target position P1 is a position where the tractor-side hitch frame 110 faces the implement-side hitch frame 210, and after that, the tractor 100 only needs to move backward linearly.
[0127] When the tractor 100 is not at the forward target position P1, the process of step S110 is repeated. When the tractor 100 is at the forward target position P1, the tractor 100 is once stopped there, the height position of the tractor-side hitch frame 110 is lowered to a position lower than the height position of the implement-side hitch frame 210, and then it moves linearly by a preset specified distance L1 (step S112).
[0128] As illustrated in FIG. 8, the preset specified distance L1 is the distance from the forward position P1 to the position (target point) where the tractor-side hitch frame 110 and the implement-side hitch frame 210 are connected. Usually, L1 is set to about 50 cm to 1 m.
[0129] In step S112, calibration using the detailed target 222 is continuously performed, the distance between the tractor-side hitch frame 110 and the implement-side hitch frame 210 is measured, and the movement of the distance L1 is precisely controlled. Here, it is also possible to finely adjust the direction of the tractor 100.
[0130] As a result of step S112, the positions of the tractor-side hitch frame 110 and the implement-side hitch frame 210 in the horizontal direction match. After that, the tractor-side hitch frame 110 is raised, and the tractor-side hitch frame 110 and the implement-side hitch frame 210 are coupled.
[0131] Next, the spline shaft 117 is moved backward, and the spline shaft 117 is coupled to the spline sleeve 212. Then, locking is performed by the hook 119. In this way, the tractor 100 and the implement 200 are coupled.
[0132] A contact sensor that detects contact between the tractor side hitch frame 110 and the work implement side hitch frame 210 may be disposed on the tractor side hitch frame 110, and reverse movement of the tractor 100 may be stopped when contact between the two is detected.
[0133] (superiority) By using two types of orientation targets, it is possible to measure the relationship between the position and orientation of the tractor relative to the implement with high accuracy even when using an inexpensive camera for orientation, which allows for reliable and stable automatic coupling of the tractor to the implement.
[0134] (others) Figure 10 shows an example of a coded target. In this case, the circle in the center is the orientation point. The position of the curvature center of the curved display is set to be the orientation point. The target can be identified by the display pattern of the design.
[0135] When both the global target and the detailed target can be recognized by image recognition, it is also possible to perform orientation using both the global target and the detailed target.
[0136] 6 and 10 show examples of location targets divided into two stages according to distance, but location targets may be divided into three or more stages. For example, a configuration is possible in which finer detailed targets that can be image-recognized and located at the closest distance are arranged, thereby improving the accuracy of location at the closest distance.
[0137] The present invention can be used not only for agricultural machinery but also for aligning a moving body with an object that the moving body is connected to or comes into contact with. For example, the present invention can be used for aligning a moving body that transports a load with the load. The present invention can also be used in a technology for determining the relative positions of a moving body and another moving body, or a technology for aligning the relative positions.
[0138] A stereo camera can also be used as the camera. In this case, the relationship between the positions and postures of the tractor and the working machine can be obtained by stereo photogrammetry. Calibration is also necessary in this case, and the present invention is utilized. Therefore, even when a stereo camera is used, the technology described in this specification can be utilized. Note that it is also conceivable to construct a stereo camera system with an increased baseline length using two cameras.
[0139] For example, the working machine 200 may be placed indoors or in a location with a roof. If there is a roof, GPS cannot be used. In this case, an IMU (inertial measurement unit) is mounted on the tractor 100, and the measured values are used to perform autonomous movement of the tractor 100 to the starting point PA in FIG. 8.
[0140] In this case, the position where the working machine 200 was detached during the previous operation is memorized, and autonomous driving to the starting point PA is performed by inertial guidance using the IMU. At this time, even if an error occurs, the accurate position of the starting point PA with respect to the working machine 200 can be specified by performing calibration using the global targets 221a and 221b.
Explanation of Reference Numerals
[0141] 100...Tractor, 110...Tractor side hitch frame, 111...Right lower support part, 111a...Right lower support pin, 112...Left support part, 112a...Left lower support pin, 113...Fork assembly, 114...Electric cylinder, 115...Drive arm, 116...Frame structure, 117...Spline shaft, 118...Electric cylinder, 119...Hook, 141...Right lower link, 141a...Support hole, 142...Left lower link, 142a...Support hole, 143...Upper support arm, 144...Right lift rod, 145...Left lift rod, 146...Right lift rod drive part, 147...Left lift rod drive part, 148...PTO shaft, 160...Connector, 161, 162...pipe for transmitting hydraulic pressure, 163...cable for electrical wiring, 170...linear bearing retaining plate, 171...linear bearing, 172...sliding shaft, 173...bearing retaining plate, 190...upper support part, 200...work implement, 210...hitch frame on work implement side, 212...spline sleeve, 213...upper fixing member, 214, 215...lower fixing member, 218...opening, 219...part with receiving connector inside, 220...target part for positioning, 221a...global target, 221b...global target, 222...detailed target, 231, 232...connection port for connecting hydraulic transmission pipe, 233...connector for electrical wiring.
Claims
1. An agricultural machine connected by a self-propelled mobile body, a connection mechanism with the mobile body, and a first type of calibration target and a second type of calibration target arranged at a position facing the mobile body when connected to the mobile body are provided, the first type of calibration target is for calibration from a relatively long distance, the second type of calibration target is for calibration from a relatively short distance, the first type of calibration target has two parts separated left and right, and the second type of calibration target is arranged at a position between the two first type of calibration targets. An agricultural machine.
2. The first type of calibration target and the second type of calibration target have a plurality of calibration points, the display constituting the calibration points of the first type of calibration target is larger than the display constituting the calibration points of the second type of calibration target, The number of calibration points of the second type of calibration target is larger than the number of calibration points of the first type of calibration target. The agricultural machine according to claim 1.
3. One or both of the first type of calibration target and the second type of calibration target have an identification surface facing the direction of the depression angle. The agricultural machine according to claim 1 or 2.
4. An agricultural machine system including the agricultural machine according to any one of claims 1 to 3, a camera, which can be connected to the agricultural machine and is a self-propelled mobile body is included, the camera shoots the second type of calibration target from the front, captures the second type of calibration target at the center of the camera's shooting screen, and in a state where the second type of calibration target can be identified, the first type of calibration target is set in a field of view outside the shooting range. An agricultural machine system.
5. An agricultural machine connection method for connecting a self-propelled mobile body to an agricultural machine, the mobile body is equipped with a camera for photographing the agricultural machine, the agricultural machine is equipped with a connection mechanism with the mobile body, a first type of calibration target and a second type of calibration target, the first type of calibration target is for calibration from a relatively long distance, the second type of calibration target is for calibration from a relatively short distance, the first type of calibration target has two parts separated left and right, and the second type of calibration target is arranged at a position between the two first type of calibration targets, At a distance where the image recognition of the calibration target of the second type by the camera is not possible, the position and orientation of the moving body with respect to the agricultural machine are detected by performing image recognition of the calibration target of the first type by the camera. Based on the position and orientation of the moving body with respect to the agricultural machine, a path for the moving body to approach the agricultural machine is set. The moving body is moved to the agricultural machine according to the set path. By the movement, when the moving body approaches the agricultural machine, the calibration target of the first type goes out of the shooting range of the camera, and image recognition of the calibration target of the second type by the camera becomes possible. Thereafter, the distance between the moving body and the agricultural machine is measured by the calibration target of the second type. An agricultural machine connection method for connecting the moving body to the agricultural machine by adjusting the distance between the moving body and the agricultural machine.
6. Movement control of the moving body along the set path is performed. In the process of the movement control, calibration of the camera using the calibration target of the first type and / or the calibration target of the second type is performed. Based on the calibration, the actual movement path of the moving body is calculated. The set path and the actual movement path are compared. The agricultural machine connection method according to claim 5, wherein the movement path is set again based on the result of the comparison.
7. A first stage where the distance between the moving body and the agricultural machine is relatively far, A second stage where the distance between the moving body and the agricultural machine is relatively close exist, In the first stage, calibration of the camera using the calibration target of the first type is performed. In the second stage, calibration of the camera using the calibration target of the second type is performed. The agricultural machine connection method according to claim 5 or 6, wherein the number of calibration points in the calibration in the second stage is larger and the calibration accuracy is higher than the calibration in the first stage.
8. A program for causing a computer to execute control for connecting a self-propelled moving body to an agricultural machine, The moving body includes a camera that photographs the agricultural machine. The agricultural machine includes a connection mechanism with the moving body, a calibration target of the first type, and a calibration target of the second type. The calibration target of the first type is for calibration from a relatively long distance. The calibration target of the second type is for calibration from a relatively short distance. The first type of calibration target is divided into two parts and arranged on the left and right, and the second type of calibration target is arranged at a position between the two first type of calibration targets. to the computer When the image recognition of the second type of calibration target by the camera cannot be performed at a distance, the position and orientation of the moving body relative to the agricultural machine are detected by image recognition of the first type of calibration target by the camera, and Based on the position and orientation of the moving body relative to the agricultural machine, setting a path for the moving body to approach the agricultural machine, and Controlling the movement of the moving body to the agricultural machine according to the set path, and By the movement control, when the moving body approaches the agricultural machine, the first type of calibration target moves out of the shooting range of the camera, and the image recognition of the second type of calibration target by the camera becomes possible. Thereafter, measuring the distance between the moving body and the agricultural machine by the second type of calibration target, and Connecting the moving body to the agricultural machine by adjusting the distance between the moving body and the agricultural machine A program to execute.
Citation Information
Patent Citations
Coupling device of working machine in tractor
JP1997103109A
Method and system for automatic marker registration
JP2010078466A
Work vehicle and travel management system therefor
JP2021106558A
Docking device of working machine for tractor
KR102062994B1