Field work vehicle

Abstract

To provide a field work vehicle with which a distance between a machine body and an obstacle is easy to be accurately estimated when a lower end portion of the obstacle is hidden by a plant in a field.SOLUTION: A field work vehicle includes: a camera 40 that images the periphery of a machine body to generate an image; a height acquisition unit that acquires a plant height being a height of a plant T of a field; a specification unit that specifies an obstacle region F1 being a region indicating an obstacle Q in the image; and an estimation unit that estimates a distance between the machine body and the obstacle Q on the basis of the plant height and the obstacle region F1.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a farm work vehicle equipped with a camera that captures images of the area around the vehicle and generates images. [Background technology]

[0002] A known example of such a farm work vehicle is described in Patent Document 1. This farm work vehicle (referred to as a "tractor" in Patent Document 1) is equipped with multiple cameras for capturing images of the area around the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-92445 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 does not describe detecting obstacles around the aircraft based on images captured by a camera. Here, if the obstacle is, for example, a person, it is conceivable to calculate the inclination of a line connecting the camera and the person's feet relative to the vertical based on the image captured by the camera, and to estimate the horizontal distance between the aircraft and the person based on the inclination.

[0005] More specifically, the product of the tangent of the tilt (angle) and the height of the camera corresponds to the horizontal distance between the aircraft (more specifically, the camera) and the person. If the height of the camera is known, this can be used to estimate the distance between the aircraft and the person.

[0006] However, if the person's feet are hidden by vegetation in the field (e.g., planted culms), it can be difficult to accurately calculate the tilt based on the image captured by the camera, which can result in a situation where the distance between the drone and the person cannot be accurately estimated.

[0007] In this way, when the bottom of an obstacle (for example, a person's feet) is hidden by plants in the field, it tends to be difficult to accurately estimate the distance between the aircraft and the obstacle.

[0008] An object of the present invention is to provide a farm work vehicle that makes it easy to accurately estimate the distance between the vehicle body and an obstacle when the lower end of the obstacle is hidden by plants in the field. [Means for solving the problem]

[0009] The present invention is characterized by comprising a camera that captures an image of the surroundings of the vehicle and generates an image, a height acquisition unit that acquires a plant height that is the height of plants in a field, an identification unit that identifies an obstacle area that is an area showing an obstacle in the image, and an estimation unit that estimates a distance between the vehicle and the obstacle based on the plant height and the obstacle area. an angle acquisition unit that acquires an angle-related value that is a value related to the inclination of a line connecting the camera and a portion of the obstacle that corresponds to the lower end of the obstacle area with respect to the vertical direction, and the estimation unit that estimates the distance between the aircraft and the obstacle based on the angle-related value and a value obtained by subtracting the plant height from the camera height. The reason is that

[0010] According to this configuration, the distance between the aircraft and the obstacle is estimated based on the plant height. As a result, for example, when the bottom of the obstacle is hidden by plants in the field, the distance between the aircraft and the obstacle can be accurately estimated by calculating the tilt of the line connecting the camera and the part of the obstacle that corresponds to the bottom of the obstacle area with respect to the vertical direction, and calculating the product of the tangent of the tilt (angle) and the value obtained by subtracting the plant height from the camera height.

[0011] That is, with this configuration, it is possible to realize a field work vehicle that can easily estimate the distance between the vehicle body and an obstacle with high accuracy when the lower end of the obstacle is hidden by plants in the field.

[0012]

[0013] Also, According to this configuration, since the angle-related value is acquired, it is easy to reliably calculate the tangent of the slope (angle) of the line connecting the camera and the part of the obstacle corresponding to the lower end of the obstacle area with respect to the vertical direction. Then, for example, by calculating the product of the tangent and the value obtained by subtracting the plant height from the camera height, it is possible to accurately estimate the distance between the aircraft and the obstacle.

[0014] Another feature of the present invention is a method for controlling a field inspection system including a camera that captures an image of the surroundings of a vehicle and generates an image, a height acquisition unit that acquires a plant height that is the height of plants in a field, an identification unit that identifies an obstacle area that is an area indicating an obstacle in the image, and an estimation unit that estimates a distance between the vehicle and the obstacle based on the plant height and the obstacle area, The system further includes a determination unit that, when the identification unit identifies the obstacle area, executes a position determination process to determine whether the obstacle is located in an area in the field where plants exist, and when the position determination process determines that the obstacle is located in an area in the field where plants exist, the estimation unit estimates the distance between the aircraft and the obstacle based on the plant height and the obstacle area, and when the position determination process determines that the obstacle is not located in an area in the field where plants exist, the estimation unit estimates the distance between the aircraft and the obstacle based on the obstacle area without based on the plant height. The thing is .

[0015] According to this configuration, the distance between the aircraft and the obstacle is estimated based on the plant height. As a result, for example, when the bottom of the obstacle is hidden by plants in the field, the distance between the aircraft and the obstacle can be accurately estimated by calculating the tilt of the line connecting the camera and the part of the obstacle that corresponds to the bottom of the obstacle area with respect to the vertical direction, and calculating the product of the tangent of the tilt (angle) and the value obtained by subtracting the plant height from the camera height. That is, with this configuration, it is possible to realize a field work vehicle that can easily estimate the distance between the vehicle body and an obstacle with high accuracy when the lower end of the obstacle is hidden by plants in the field. Also, According to this configuration, it is possible to realize a configuration in which the distance between the aircraft and an obstacle can be estimated in an appropriate manner depending on whether the obstacle is located in an area of ​​the field where plants exist.

[0016] Furthermore, in the present invention, it is preferable that a map acquisition unit is provided that acquires an area map showing unworked areas in the field, and the determination unit executes the position determination process based on the area map.

[0017] This configuration makes it easy to accurately determine whether an obstacle is located in an area of ​​the field where plants exist, which makes it easy to appropriately use different methods for estimating the distance between the aircraft and the obstacle.

[0018] Furthermore, in the present invention, it is preferable that the determination unit executes the position determination process based on the image.

[0019] According to this configuration, for example, the position determination process can be performed by determining whether or not there are plants near an obstacle in an image through image analysis, which makes it easier to accurately determine whether or not the obstacle is located in an area of ​​the field where plants exist.

[0020] Furthermore, in the present invention, it is preferable that a detection unit is provided that detects the three-dimensional position of an object around the aircraft, and the height acquisition unit acquires the plant height based on the detection result by the detection unit.

[0021] According to this configuration, it is possible to realize a configuration in which the plant height is automatically acquired, thereby realizing a configuration in which information regarding the plant height (for example, a value indicating the plant height) does not need to be manually input.

[0022] Another feature of the present invention is a method for controlling a field inspection system including a camera that captures an image of the surroundings of a vehicle and generates an image, a height acquisition unit that acquires a plant height that is the height of plants in a field, an identification unit that identifies an obstacle area that is an area indicating an obstacle in the image, and an estimation unit that estimates a distance between the vehicle and the obstacle based on the plant height and the obstacle area, a reception unit that receives a manual operation input regarding the plant height, and the height acquisition unit acquires the plant height based on the manual operation input received by the reception unit The thing is .

[0023] According to this configuration, the distance between the aircraft and the obstacle is estimated based on the plant height. As a result, for example, when the bottom of the obstacle is hidden by plants in the field, the distance between the aircraft and the obstacle can be accurately estimated by calculating the tilt of the line connecting the camera and the part of the obstacle that corresponds to the bottom of the obstacle area with respect to the vertical direction, and calculating the product of the tangent of the tilt (angle) and the value obtained by subtracting the plant height from the camera height. That is, with this configuration, it is possible to realize a field work vehicle that can easily estimate the distance between the vehicle body and an obstacle with high accuracy when the lower end of the obstacle is hidden by plants in the field. Also, According to this configuration, it is possible to obtain the plant height even when a sensor for detecting the plant height is not provided, which makes it easier to avoid a situation in which the provision of a sensor for detecting the plant height would increase manufacturing costs.

[0024] Furthermore, in the present invention, it is preferable to further include an aircraft control unit that controls the operation of the aircraft based on the distance estimated by the estimation unit.

[0025] This configuration allows the vehicle to be automatically controlled according to the distance between the vehicle and an obstacle, thereby enabling the vehicle to stop moving when the distance between the vehicle and an obstacle is relatively short, for example. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. [Figure 2] FIG. 2 is a block diagram showing a configuration related to a control unit. [Figure 3] FIG. 10 is a diagram showing a flow of obstacle detection by an identification unit. [Figure 4] FIG. 10 is a diagram showing an already worked area and an unworked area. [Figure 5] 10 is a flowchart of a distance estimation flow. [Figure 6] FIG. 10 is a diagram illustrating a first estimation method. [Figure 7] FIG. 10 is a diagram illustrating a second estimation method. DETAILED DESCRIPTION OF THE INVENTION

[0027] An embodiment of the present invention will be described with reference to the drawings. In the following description, unless otherwise specified, the direction of arrow F in the drawings will be referred to as "front" and the direction of arrow B as "rear." Furthermore, the direction of arrow U in the drawings will be referred to as "up" and the direction of arrow D as "down."

[0028] [Overall configuration of the combine] As shown in Figure 1, a standard combine harvester 1 (corresponding to the "field work vehicle" according to the present invention) is equipped with a harvesting section H, a crawler-type traveling device 11, a driving section 12, a threshing device 13, a grain tank 14, a conveying section 16, a grain discharge device 18, and a satellite positioning module 80.

[0029] The traveling device 11 is provided at the bottom of the combine harvester 1. The traveling device 11 is driven by power from an engine (not shown) mounted on the combine harvester 1. The combine harvester 1 can travel by using the traveling device 11.

[0030] The driving section 12, threshing device 13, and grain tank 14 are provided above the traveling device 11. An operator can ride in the driving section 12 to monitor the operation of the combine harvester 1. The operator may also monitor the operation of the combine harvester 1 from outside the combine harvester 1.

[0031] The driver's section 12 has a driver's seat 12a and a cabin 12b. The driver's seat 12a is provided inside the cabin 12b. An operator can sit in the driver's seat 12a.

[0032] The grain discharge device 18 is provided on the upper side of the grain tank 14. In addition, the satellite positioning module 80 is attached to the upper surface of the driving section 12.

[0033] The harvesting unit H is provided at the front of the combine 1. The transport unit 16 is provided behind the harvesting unit H. The harvesting unit H, together with the transport unit 16, is configured to be able to swing up and down around a swing axis (not shown) extending in the left-right direction of the machine body. This allows the harvesting unit H to move up and down. The swing axis is located at the rear end of the transport unit 16. The harvesting unit H also includes a reaping device 15 and a reel 17.

[0034] The reaping device 15 reaps crops P in the field. The crops P may be, for example, rice, although there is no particular limitation. The reel 17 rakes in the crops P to be harvested while rotating around a reel axis 17b that runs along the left-right direction of the machine body. The crops P reaped by the reaping device 15 are sent to the conveying section 16.

[0035] With this configuration, the harvesting section H harvests the crops P in the field. The combine 1 is capable of reaping travel, traveling by the traveling device 11 while reaping the crops P in the field with the reaping device 15.

[0036] The crops P harvested by the harvesting section H are transported to the rear of the machine body by the transport section 16. As a result, the crops P are transported to the threshing device 13.

[0037] The crop P is threshed in the threshing device 13. The harvested product (grains) obtained by the threshing process is stored in a grain tank 14. The harvested product stored in the grain tank 14 is discharged outside the machine by a grain discharge device 18 as needed.

[0038] The combine harvester 1 may be configured to be capable of automatic travel in a farm field, or may be configured not to be capable of automatic travel. When the combine harvester 1 is configured to be automatic travel, for example, the combine harvester 1 may automatically travel along a generated route (not shown).

[0039] [Obstacle detection] 1 and 2, the combine harvester 1 is equipped with a camera 40. The camera 40 is attached to the front end portion on the upper surface of the driver's section 12. The camera 40 faces downward and forward.

[0040] The camera 40 generates an image by capturing an image of the periphery of the body of the combine harvester 1. More specifically, the camera 40 captures an image in front of the body of the combine harvester 1 and generates an image.

[0041] In this way, the combine harvester 1 is equipped with a camera 40 that captures images of the surroundings of the machine body and generates images.

[0042] 1 and 2, the combine harvester 1 includes a control unit 20. As shown in FIG.

[0043] As shown in Fig. 2, an image (captured image) generated by the camera 40 is sent to the identification unit 21. The identification unit 21 detects an obstacle Q (see Fig. 1) around the aircraft based on the image captured by the camera 40. At this time, the identification unit 21 detects the obstacle Q around the aircraft by using a neural network trained using deep learning.

[0044] 3 shows a flow of obstacle detection by the identification unit 21. In the following, the detection of an obstacle Q by the identification unit 21 will be described, assuming that the object to be detected by the identification unit 21 is a person.

[0045] 3, the pixel value of each pixel included in the image captured by the camera 40 is input to the identification unit 21. Then, data indicating an estimation result (detection result) is output from the identification unit 21. This data includes the presence area of ​​the person, which is the obstacle Q, and the estimation probability thereof.

[0046] In the estimation result in FIG. 3, an obstacle area F1, which is an area where a person (obstacle Q) exists, is shown by a rectangular frame. The obstacle area F1 is an area that indicates the obstacle Q. An estimation probability is linked to the obstacle area F1. The obstacle area F1 is also defined by four corner points. The coordinate positions of these four corner points in the captured image are also included in the estimation result. Note that if a detection object is not estimated in the captured image, the obstacle area F1 is not output and the estimation probability is zero.

[0047] With the configuration described above, the identification unit 21 identifies the obstacle region F1 in the image generated by the camera 40. In this way, the identification unit 21 detects the obstacle Q.

[0048] In this way, the combine harvester 1 is provided with an identification unit 21 that identifies the obstacle area F1, which is an area indicating the obstacle Q, in the image.

[0049] The control unit 20 and each element included in the control unit 20, such as the specifying unit 21, may be a physical device such as a microcomputer, or may be a functional unit in software.

[0050] [Position Determination Processing] As shown in FIG. 2, the control unit 20 includes a position calculation unit 22, a map acquisition unit 23, a direction calculation unit 24, and a determination unit 25.

[0051] The satellite positioning module 80 receives GPS signals from the artificial satellites GS (see FIG. 1) used in the GPS (Global Positioning System). Then, as shown in FIG. 2, the satellite positioning module 80 sends positioning data indicating the vehicle position of the combine harvester 1 to the position calculation unit 22 based on the received GPS signals.

[0052] However, the present invention is not limited to this. The satellite positioning module 80 does not have to use GPS. For example, the satellite positioning module 80 may use GNSS (GLONASS, Galileo, Michibiki, BeiDou, etc.) other than GPS.

[0053] The position calculation unit 22 calculates the position coordinates of the combine harvester 1 over time based on the positioning data output by the satellite positioning module 80. The calculated position coordinates of the combine harvester 1 over time are sent to the map acquisition unit 23, the orientation calculation unit 24, and the determination unit 25.

[0054] The map acquisition unit 23 calculates the worked area SA and the unworked area CA as shown in FIG. 4 based on the time-varying position coordinates of the combine harvester 1 received from the position calculation unit 22.

[0055] More specifically, the map acquisition unit 23 acquires data indicating the outline of the farm field. The data may be acquired, for example, based on the position coordinates of the combine harvester 1 over time when the combine harvester 1 makes a circular run in the outermost peripheral region of the farm field. Then, while the combine harvester 1 is performing a reaping run (work run) in the farm field, the map acquisition unit 23 calculates the reaping run trajectory of the combine harvester 1 in the farm field based on the position coordinates of the combine harvester 1 over time received from the position calculation unit 22.

[0056] The map acquisition unit 23 calculates the area where the combine harvester 1 has performed the reaping travel as the worked area SA over time, based on the reaping travel trajectory of the combine harvester 1. The map acquisition unit 23 also calculates the unworked area CA over time based on data indicating the outline of the field and the worked area SA. At this time, the map acquisition unit 23 calculates the part of the field that is not the worked area SA as the unworked area CA.

[0057] In this way, the map acquisition unit 23 generates an area map indicating the worked area SA and the unworked area CA in the field over time. As a result, the map acquisition unit 23 acquires this area map.

[0058] In this way, the combine harvester 1 is provided with a map acquisition unit 23 that acquires an area map showing the unworked area CA in the farm field.

[0059] In this embodiment, the already worked area SA is an area where there are no uncut crops P. The unworked area CA is an area where there are uncut crops P.

[0060] As shown in FIG. 2, the region map acquired by the map acquisition unit 23 is sent to the determination unit 25.

[0061] 2, the combine harvester 1 is also equipped with an inertial measurement unit 81. The inertial measurement unit 81 detects the angular velocity of the yaw angle of the combine harvester 1's body and the acceleration in three mutually orthogonal axial directions over time. The detection results by the inertial measurement unit 81 are sent to the orientation calculation unit 24.

[0062] The orientation calculation unit 24 receives the position coordinates of the combine harvester 1 from the position calculation unit 22. Then, the orientation calculation unit 24 calculates the body orientation of the combine harvester 1 based on the detection results from the inertial measurement unit 81 and the position coordinates of the combine harvester 1.

[0063] More specifically, while the combine harvester 1 is traveling, the orientation calculation unit 24 first calculates an initial vehicle orientation based on the current position coordinates of the combine harvester 1 and the position coordinates of the combine harvester 1 at the point where it was traveling immediately before. Next, when the combine harvester 1 has traveled for a certain period of time after the initial vehicle orientation is calculated, the orientation calculation unit 24 calculates the amount of change in the vehicle orientation by integrating the angular velocity detected by the inertial measurement unit 81 during that certain period of travel.

[0064] Then, the orientation calculation unit 24 updates the calculation result of the aircraft orientation by adding the calculated change in aircraft orientation to the initial aircraft orientation. After that, the change in aircraft orientation is similarly calculated at regular time intervals, and the calculation result of the aircraft orientation is successively updated.

[0065] With the above configuration, the orientation calculation unit 24 calculates the body orientation of the combine harvester 1. The calculation result by the orientation calculation unit 24 is sent to the determination unit 25.

[0066] When the identification unit 21 identifies the obstacle area F1, as shown in Fig. 2, the identification unit 21 sends a predetermined signal to the determination unit 25. This signal indicates that the identification unit 21 has identified the obstacle area F1. Upon receiving this signal, the determination unit 25 executes a position determination process. The position determination process is a process for determining whether or not the obstacle Q is located in an area in the field where the plant T exists.

[0067] In this way, the combine 1 is equipped with a determination unit 25 that performs a position determination process to determine whether the obstacle Q is located in an area in the field where a plant T exists when the identification unit 21 identifies the obstacle area F1.

[0068] The crop P is a specific example of the plant T. However, the specific example of the plant T is not limited to the crop P. The plant T may be, for example, a weed.

[0069] In the position determination process, the determination unit 25 determines whether the obstacle Q is located in an area in the field where the plant T is present based on the above-mentioned area map, the current position coordinates of the combine 1, and the current body orientation of the combine 1.

[0070] That is, the determination unit 25 executes the position determination process based on the area map.

[0071] For example, Fig. 4 shows a first position P1 and a second position P2. When the combine harvester 1 is located at the first position P1, an unworked area CA exists in front of the combine harvester 1. If the identification unit 21 identifies an obstacle area F1 at this time, the determination unit 25 determines through position determination processing that the obstacle Q is located in an area (unworked area CA) in the field where a plant T (crop P) exists.

[0072] 4, when the combine harvester 1 is located at the second position P2, there is no unworked area CA in front of the combine harvester 1. If the identification unit 21 identifies an obstacle area F1 at this time, the determination unit 25 determines through position determination processing that the obstacle Q is not located in an area (unworked area CA) in the field where a plant T (crop P) is present.

[0073] [Plant height] As shown in Fig. 1, the combine harvester 1 is equipped with a detection unit 41. The detection unit 41 is attached to the front end of the upper surface of the driving unit 12. The detection unit 41 is disposed adjacent to the camera 40. The detection unit 41 faces downward and forward.

[0074] The detection unit 41 detects the three-dimensional position of an object around the body of the combine harvester 1. More specifically, the detection unit 41 detects the three-dimensional position of an object in front of the body of the combine harvester 1. This enables the detection unit 41 to detect the three-dimensional position of the top end of the plant T (crop P).

[0075] In this way, the combine harvester 1 is equipped with a detection unit 41 that detects the three-dimensional position of an object around the machine body.

[0076] The detection unit 41 in this embodiment is a LiDAR (Light Detection And Ranging) sensor, which is a measurement device using a ToF (Time of flight) measurement method. Note that the measurement method of the detection unit 41 is not limited to the ToF measurement method, and may be a stereo matching measurement method or the like.

[0077] As shown in Fig. 2, the combine harvester 1 includes a receiving unit 42. The receiving unit 42 is configured to receive manual input related to plant height. Note that the plant height is the height of the plants T (crops P) in the field.

[0078] Thus, the combine harvester 1 is provided with the reception unit 42 that receives manual operation input related to plant height.

[0079] The receiving unit 42 may be, for example, a touch panel, a keyboard, a mouse, etc. The information that the receiving unit 42 receives by manual input is not particularly limited, but may be, for example, information indicating the species (rice, soybean, wheat, etc.) or variety of the plant T (crop P).

[0080] 2, the control unit 20 has a height acquisition unit 26. The height acquisition unit 26 acquires the detection result by the detection unit 41. The height acquisition unit 26 also acquires information received by the reception unit 42 through manual input. The height acquisition unit 26 then calculates the plant height based on the detection result by the detection unit 41 and the information received by the reception unit 42 through manual input. In this way, the height acquisition unit 26 acquires the plant height.

[0081] As described above, the combine harvester 1 includes a height acquisition unit 26 that acquires the plant height, which is the height of the plants T in the field. The height acquisition unit 26 acquires the plant height based on the detection result by the detection unit 41. The height acquisition unit 26 also acquires the plant height based on the manual operation input received by the reception unit 42.

[0082] [Distance estimation] As shown in Fig. 2, the control unit 20 has an estimation unit 27. The estimation unit 27 estimates the distance between the body of the combine harvester 1 and an obstacle Q. The control unit 20 is configured to perform processing related to estimation of the distance between the body and the obstacle Q in accordance with the distance estimation flow shown in Fig. 5 when the combine harvester 1 is traveling in a field. Note that this distance estimation flow may be executed when the combine harvester 1 is traveling manually or automatically.

[0083] When this distance estimation flow starts, first, the processing of step S01 is executed. In step S01, it is determined whether or not the identification unit 21 has identified an obstacle area F1. If the identification unit 21 has not identified the obstacle area F1 ("No" in step S01 of FIG. 5), this distance estimation flow is temporarily terminated. If the identification unit 21 has identified the obstacle area F1 ("Yes" in step S01 of FIG. 5), the processing proceeds to step S02.

[0084] In step S02, a position determination process is executed by the determination unit 25. The determination result of the position determination process is sent from the determination unit 25 to the estimation unit 27 (see FIG. 2).

[0085] If the position determination process determines that the obstacle Q is located in an area of ​​the field where the plants T exist ("Yes" in step S02 of FIG. 5), the process proceeds to step S03. If the position determination process determines that the obstacle Q is not located in an area of ​​the field where the plants T exist ("No" in step S02 of FIG. 5), the process proceeds to step S04.

[0086] In step S03, the estimation unit 27 estimates the distance between the aircraft and the obstacle Q using a first estimation method. The first estimation method is a method of estimating the distance between the aircraft and the obstacle Q based on the plant height and the obstacle region F1. The first estimation method will be described in detail below.

[0087] As shown in Fig. 2, the control unit 20 has an angle acquisition unit 28. The angle acquisition unit 28 acquires an angle-related value. The angle-related value is a value related to the inclination of a reference line Y (see Figs. 6 and 7) with respect to the vertical direction. The reference line Y is a line connecting the camera 40 and a portion of the obstacle Q that corresponds to the lower end of the obstacle area F1.

[0088] In this way, the combine 1 is equipped with an angle acquisition unit 28 that acquires an angle-related value, which is a value related to the inclination of the line connecting the camera 40 and the portion of the obstacle Q corresponding to the lower end of the obstacle area F1 relative to the vertical direction.

[0089] In this embodiment, the angle-related value is the inclination of the reference line Y with respect to the vertical direction. However, the present invention is not limited to this. The angle-related value may be, for example, the inclination (depression angle) of the reference line Y with respect to the horizontal direction, or the inclination of the reference line Y with respect to the orientation of the camera 40.

[0090] In this embodiment, the angle acquisition unit 28 stores corresponding angle data, which is data indicating the correspondence between each pixel of an image captured by the camera 40 and the corresponding angle. The corresponding angle is the tilt of the direction corresponding to one pixel in the captured image relative to the orientation of the camera 40. For example, the corresponding angle of a pixel located at the center of the captured image is 0 degrees. Furthermore, the corresponding angle of a pixel increases as the distance from the center of the captured image to the pixel increases. The corresponding angle data can be determined experimentally.

[0091] In this embodiment, the angle acquisition unit 28 stores the tilt of the orientation of the camera 40 relative to the vertical direction. In this embodiment, the orientation of the camera 40 is fixed.

[0092] In the first estimation method, first, information indicating the pixel corresponding to the bottom end of the obstacle area F1 in the captured image is sent from the identification unit 21 to the angle acquisition unit 28. Next, the angle acquisition unit 28 calculates the inclination of the reference line Y with respect to the vertical direction based on the information, the corresponding angle data described above, and the inclination of the orientation of the camera 40 with respect to the vertical direction. In this way, the angle acquisition unit 28 acquires the inclination of the reference line Y with respect to the vertical direction.

[0093] Next, the estimation unit 27 acquires the plant height from the height acquisition unit 26 and acquires the inclination of the reference line Y with respect to the vertical direction from the angle acquisition unit 28. Then, the estimation unit 27 estimates the distance between the body of the combine harvester 1 and the obstacle Q by calculating the product of the tangent of the inclination (angle) of the reference line Y with respect to the vertical direction and the value obtained by subtracting the plant height from the height of the camera 40. In other words, the estimated value of the distance between the body and the obstacle Q is calculated by calculating the product of the tangent of the inclination (angle) of the reference line Y with respect to the vertical direction and the value obtained by subtracting the plant height from the height of the camera 40. The height of the camera 40 is stored in advance in the estimation unit 27.

[0094] Through the processing described above, the distance between the aircraft and the obstacle Q is estimated using the first estimation method.

[0095] As described above, the combine harvester 1 is provided with an estimation unit 27 that estimates the distance between the machine body and the obstacle Q based on the plant height and the obstacle region F1. The estimation unit 27 also estimates the distance between the machine body and the obstacle Q based on a value obtained by subtracting the plant height from the height of the camera 40 and an angle-related value. Furthermore, when it is determined by the position determination process that the obstacle Q is located in an area in the field where the plants T are present, the estimation unit 27 estimates the distance between the machine body and the obstacle Q based on the plant height and the obstacle region F1.

[0096] 6, an obstacle Q is located in an area in the field where a plant T (crop P) exists. In this case, the distance between the aircraft and the obstacle Q is estimated by the first estimation method.

[0097] In the example shown in FIG. 6, the lower half of the person's body, which is an obstacle Q, is hidden by a plant T (crop P). In this case, the obstacle region F1 is a frame that surrounds only the unhidden part of the person. Therefore, the part of the person that corresponds to the lower end of the obstacle region F1 is the person's waist. Therefore, the reference line Y in this example is a line that connects the camera 40 and the person's waist.

[0098] In the example shown in Fig. 6, the inclination of the reference line Y with respect to the vertical direction is A1. The height of the camera 40 is H1. The height of the plant is H2. The distance between the aircraft (more specifically, the camera 40) and the obstacle Q is D1.

[0099] in this case, D1=(H1-H2)×tanA1 In this way, the distance between the aircraft and obstacle Q is estimated.

[0100] As shown in Fig. 2, the control unit 20 has an aircraft control unit 29. When the distance between the aircraft and obstacle Q is estimated by the first estimation method in step S03 shown in Fig. 5, the estimated distance is sent from the estimation unit 27 to the aircraft control unit 29. Then, the processing proceeds to step S05.

[0101] In step S05, the machine control unit 29 controls the operation of the machine based on the distance estimated by the estimation unit 27. While not particularly limited, the machine control unit 29 may, for example, stop the travel of the machine when the distance estimated by the estimation unit 27 is equal to or shorter than a predetermined distance. Furthermore, for example, the machine control unit 29 may control the operation of either or both of the traveling device 11 and the harvesting unit H so that the combine harvester 1 travels while avoiding the obstacle Q, depending on the distance estimated by the estimation unit 27. At this time, for example, the machine control unit 29 may raise the harvesting unit H. Furthermore, for example, the machine control unit 29 may control the machine to issue a warning using sound, light, or the like to either or both of the obstacle Q and the operator, depending on the distance estimated by the estimation unit 27.

[0102] In this way, the combine harvester 1 is provided with a machine control unit 29 that controls the operation of the machine based on the distance estimated by the estimation unit 27.

[0103] After step S05, this distance estimation flow ends for the time being.

[0104] In step S04, the estimation unit 27 estimates the distance between the aircraft and the obstacle Q using a second estimation method. The second estimation method is a method of estimating the distance between the aircraft and the obstacle Q based on the obstacle region F1, not based on the plant height. The second estimation method will be described in detail below.

[0105] In the second estimation method, first, information indicating the pixel corresponding to the bottom end of the obstacle area F1 in the captured image is sent from the identification unit 21 to the angle acquisition unit 28. Next, the angle acquisition unit 28 calculates the inclination of the reference line Y with respect to the vertical direction based on the information, the corresponding angle data described above, and the inclination of the orientation of the camera 40 with respect to the vertical direction. In this way, the angle acquisition unit 28 acquires the inclination of the reference line Y with respect to the vertical direction.

[0106] Next, the estimation unit 27 acquires the inclination of the reference line Y with respect to the vertical direction from the angle acquisition unit 28. Then, the estimation unit 27 estimates the distance between the body of the combine 1 and the obstacle Q by calculating the product of the tangent of the inclination (angle) of the reference line Y with respect to the vertical direction and the height of the camera 40. In other words, the product of the tangent of the inclination (angle) of the reference line Y with respect to the vertical direction and the height of the camera 40 is calculated as an estimated value of the distance between the body and the obstacle Q.

[0107] Through the processing described above, the distance between the aircraft and the obstacle Q is estimated by the second estimation method.

[0108] In this way, if the position determination process determines that the obstacle Q is not located in the area of ​​the field where the plant T exists, the estimation unit 27 estimates the distance between the aircraft and the obstacle Q based on the obstacle area F1, without basing it on the plant height.

[0109] 7, the obstacle Q is not located in an area of ​​the field where the plant T (crop P) is present. In other words, the obstacle Q is located in an area of ​​the field where the plant T (crop P) is not present. In this case, the distance between the aircraft and the obstacle Q is estimated by the second estimation method.

[0110] 7, the part of the person who is obstacle Q that corresponds to the lower end of obstacle region F1 is the feet of the person. Therefore, reference line Y in this example is a line connecting camera 40 and the feet of the person.

[0111] 7, the inclination of the reference line Y with respect to the vertical direction is A2, the height of the camera 40 is H1, and the distance between the aircraft (more specifically, the camera 40) and the obstacle Q is D2.

[0112] in this case, D2=H1×tanA2 In this way, the distance between the aircraft and obstacle Q is estimated.

[0113] 5, when the distance between the aircraft and obstacle Q is estimated by the second estimation method, the estimated distance is sent from the estimation unit 27 to the aircraft control unit 29. Then, the processing proceeds to step S05. The processing in step S05 is as described above.

[0114] According to the configuration described above, the distance between the aircraft and obstacle Q is estimated based on the plant height. As a result, for example, when the lower end of obstacle Q is hidden by plants T in the field, the distance between the aircraft and obstacle Q can be accurately estimated by calculating the inclination of the line connecting camera 40 and a portion of obstacle Q that corresponds to the lower end of obstacle region F1 with respect to the vertical direction and calculating the product of the tangent of the inclination (angle) and the value obtained by subtracting the plant height from the height of camera 40.

[0115] In other words, the configuration described above makes it possible to realize a combine harvester 1 that can easily estimate the distance between the machine body and the obstacle Q with high accuracy when the lower end of the obstacle Q is hidden by plants T in the field.

[0116] Other Embodiments (1) The camera 40 may capture images not only in front of the combine 1 but also in other directions. Also, multiple cameras 40 may be provided. The multiple cameras 40 may be oriented in different directions.

[0117] (2) The information that the receiving unit 42 receives through manual input may be a value indicating the plant height (more specifically, the plant height itself).

[0118] (3) The detection unit 41 does not have to be provided. That is, the height acquisition unit 26 may acquire the plant height without relying on the detection result by the detection unit 41.

[0119] (4) The receiving unit 42 does not have to be provided. That is, the height obtaining unit 26 may obtain the plant height without relying on a manual operation input received by the receiving unit 42.

[0120] (5) The orientation of the camera 40 may be variable. In this case, the orientation of the camera 40 may be detected when the camera 40 is pointed toward a portion of the obstacle Q that corresponds to the lower end of the obstacle area F1. In this case, the detected orientation of the camera 40 corresponds to the "angle-related value" according to the present invention. Furthermore, the detected orientation of the camera 40 coincides with the direction in which the reference line Y extends. In other words, the distance between the aircraft and the obstacle Q can be estimated based on the detected orientation of the camera 40.

[0121] (6) The distance estimated by the estimation unit 27 may be the distance between any part of the vehicle and the obstacle Q. For example, the distance estimated by the estimation unit 27 may be the distance between the front end of the camera 40 in the fore-and-aft direction of the vehicle and the obstacle Q, or the distance between the front end of the harvesting unit H in the fore-and-aft direction of the vehicle and the obstacle Q. The distance between the front end of the harvesting unit H in the fore-and-aft direction of the vehicle and the obstacle Q can be calculated (estimated), for example, by subtracting the distance between the front end of the camera 40 in the fore-and-aft direction of the vehicle and the front end of the harvesting unit H from D1 shown in FIG. 6.

[0122] (7) The determination unit 25 may perform a position determination process based on an image captured by the camera 40. For example, the determination unit 25 may perform image analysis using a neural network trained using deep learning to determine whether the obstacle Q is located in an area of ​​the field where the plant T is present.

[0123] (8) When the angle-related value acquired by the angle acquisition unit 28 is the inclination (depression angle) of the reference line Y relative to the horizontal direction, the estimation unit 27 may estimate the distance between the body of the combine 1 and the obstacle Q, for example, by subtracting the plant height from the height of the camera 40 and dividing the value obtained by the tangent of the inclination (angle).

[0124] (9) A sensor that detects the length of the reference line Y (in other words, the distance between the camera 40 and the portion of the obstacle Q that corresponds to the lower end of the obstacle region F1) may be provided. In this case, the estimation unit 27 may estimate the distance between the body of the combine 1 and the obstacle Q using Pythagoras's theorem, for example, based on the value obtained by subtracting the plant height from the height of the camera 40 and the length of the reference line Y. In this case, the angle acquisition unit 28 may not be provided.

[0125] (10) The determination unit 25 may not be provided. In this case, the estimation unit 27 may always estimate the distance between the aircraft and the obstacle Q based on the plant height and the obstacle area F1, regardless of whether the obstacle Q is located in an area in the field where the plants T exist.

[0126] (11) The obstacle Q is not limited to a person. The obstacle Q may be, for example, a bird, an animal, a tree, a building, or the like.

[0127] (12) The obstacle region F1 may have any shape. For example, the obstacle region F1 may be circular or may have a shape that follows the outer shape of the obstacle Q.

[0128] The configurations disclosed in the above-described embodiments (including other embodiments, the same applies hereinafter) can be applied in combination with the configurations disclosed in other embodiments, unless a contradiction arises. Furthermore, the embodiments disclosed in this specification are merely examples, and the present invention is not limited to these, and can be modified as appropriate within the scope of the purpose of the present invention. [Industrial Applicability]

[0129] The present invention can be used not only in ordinary combine harvesters, but also in various other farm work vehicles such as head-feeding combine harvesters, tractors, rice transplanters, corn harvesters, potato harvesters, and carrot harvesters. [Explanation of symbols]

[0130] 1: Combine (field work vehicle) 21: Specific part 23: Map acquisition section 25: Judgment section 26: Height acquisition unit 27:Estimation part 28: Angle acquisition section 29: Aircraft control unit 40: Camera 41: Detection unit 42: Reception CA: Unworked area F1: Obstacle area Q: Obstacles T:Plant

Claims

1. a camera that captures images of the surroundings of the aircraft and generates images; a height acquisition unit that acquires a plant height, which is the height of a plant in a field; an identification unit that identifies an obstacle area in the image, the obstacle area being an area indicating an obstacle; an estimation unit that estimates a distance between the aircraft and the obstacle based on the plant height and the obstacle area, an angle acquisition unit that acquires an angle-related value that is a value related to the inclination of a line connecting the camera and a portion of the obstacle that corresponds to a lower end of the obstacle area with respect to a vertical direction, The estimation unit estimates the distance between the vehicle body and the obstacle based on a value obtained by subtracting the plant height from the camera height and the angle-related value.

2. A camera that captures an image of the surroundings of the aircraft and generates an image; a height acquisition unit that acquires a plant height, which is the height of a plant in a field; an identification unit that identifies an obstacle area in the image, the obstacle area being an area indicating an obstacle; an estimation unit that estimates a distance between the aircraft and the obstacle based on the plant height and the obstacle area, a determination unit that, when the identification unit identifies the obstacle area, executes a position determination process to determine whether the obstacle is located in an area in the field where plants exist, When the position determination process determines that the obstacle is located in an area in the field where plants exist, the estimation unit estimates a distance between the aircraft and the obstacle based on the plant height and the obstacle area; If the position determination process determines that the obstacle is not located in an area of ​​the field where plants exist, the estimation unit estimates the distance between the vehicle and the obstacle based on the obstacle area, without basing it on the plant height.

3. a map acquisition unit that acquires an area map showing an unworked area in the farm field; The field work vehicle according to claim 2 , wherein the determination unit executes the position determination process based on the area map.

4. The field work vehicle according to claim 2 , wherein the determining unit executes the position determining process based on the image.

5. A camera that captures an image of the surroundings of the aircraft and generates an image; a height acquisition unit that acquires a plant height, which is the height of a plant in a field; an identification unit that identifies an obstacle area in the image, the obstacle area being an area indicating an obstacle; an estimation unit that estimates a distance between the aircraft and the obstacle based on the plant height and the obstacle area, a reception unit for receiving a manual operation input regarding the plant height, The height acquisition unit acquires the plant height based on the manual operation input received by the reception unit.

6. a detection unit that detects the three-dimensional position of an object around the aircraft; The field work vehicle according to claim 1 , wherein the height acquisition unit acquires the plant height based on a detection result by the detection unit.

7. The field work vehicle according to any one of claims 1 to 5, further comprising a vehicle control unit that controls operation of the vehicle based on the distance estimated by the estimation unit.

Images