Harvesting work vehicle

Abstract

To solve such the problem that a harvesting work vehicle can automatically travel in a farm field on the basis of satellite positioning device but has a concern about that wheels of the work vehicle may step on crops because it cannot recognize a planting state of crop.SOLUTION: A harvest work vehicle is provided with a plurality of imaging devices to include part of the vehicle in an image and utilize a relative distance, corrects a three-dimensional dimension and detects a position of a crop planted in a farm field, thereby automatically travelling.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a work vehicle that is equipped with multiple imaging devices and can utilize image data from each imaging device to not only capture wide-angle images, but also measure the distance to an object and determine the arrangement of crops in a field, enabling automatic driving. [Background technology]

[0002] In the prior art, there is a technology that measures distances using a stereo camera and detects the distance to an obstacle captured in an image, and there is an automatic driving control that uses the distance interval of the image capturing device (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-174344 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology includes a distance information generation unit that generates three-dimensional point cloud data expanded in a three-dimensional coordinate system with the field scene as a single reference plane from images captured by a stereo camera unit, a field harvesting information creation unit and an obstacle estimation means, and an obstacle detection unit that detects obstacles and calculates the positional relationship between the obstacles and the aircraft based on the three-dimensional point cloud data and the estimation results by the obstacle estimation means.

[0005] However, because the height position between the stereo camera on the work vehicle and the field is not constant due to the undulations of the field, errors occur in the distance calculation, and it is not possible to draw a straight line when calculating the diagonal distance.

[0006] The present invention aims to provide a work vehicle that can prevent errors in distance calculations of the imaging device due to the undulating nature of the field by positioning the synthesized reference position of the stereo camera so that it overlaps part of the tip of the work vehicle and applying relative scale correction. [Means for solving the problem]

[0007] The first aspect of the present invention is achieved by the following technical means.

[0008] The left imaging device 100 is provided at the left end 110 at the front of the roof part above the frame structure that protects the operating unit, and the right imaging device 150 is provided at the right end 160. The central axis 101 of the imaging direction of the left imaging device 100 is inclined at an angle α1 to the left and an angle β1 downward from the traveling direction axis γ of the vehicle body, and the center of focus is located below the left side of the vehicle body. The central axis 151 of the imaging direction of the right imaging device 150 is inclined at an angle α2 to the right and an angle β2 downward from the traveling direction axis γ of the vehicle body, and the center of focus is located below the right side of the vehicle body. The vehicle width dimension is within the width dimension X of the front and rear wheels. A left imaging device 100 and a right imaging device 150 are provided on the work vehicle, and an intersection 170 of a right end line 103 of a wide angle 102 of imaging by the left imaging device 100 and a left end line 153 of a wide angle 152 of imaging by the right imaging device 150 is located behind a front end position 180 of the vehicle hood, and the front end position 180 of the vehicle hood is imaged by the left imaging device 100 and the right imaging device 150, thereby measuring a distance L1 to a front position A1, a distance L2 to a lateral position A2, and a distance L3 to a lateral position A3 by a ratio calculation of a reference dimension 181.

[0009] The second aspect of the present invention is achieved by the following technical means.

[0010] Based on the difference in the images of the soil in the field on each side of the left imaging device 100 and the right imaging device 150, the positions of the uncultivated and cultivated areas are determined, and a straight line for automatic travel is determined based on the line 200 in the cultivated area.

[0011] The third aspect of the present invention is achieved by the following technical means.

[0012] The left image pickup device 100 and the right image pickup device 150 detect the position of greenery, and a continuously connected line on the image is recognized as a crop planting line, which is collated with the width dimension X of the front and rear wheels. Automatic travel is performed so that the vehicle travels along the planting line position at a position where the front and rear wheels do not overlap. Effect of the Invention

[0013] According to the first and second aspects of the present invention, the stereo camera can be used to determine the distance to an object located diagonally from the traveling direction, and to determine parallel lines on the side.

[0014] According to the third aspect of the present invention, it is possible for the work vehicle to travel automatically without running over crops with its wheels. [Brief description of the drawings]

[0015] [Figure 1] FIG. 2 is a diagram showing the arrangement positions of the left and right imaging devices, the imaging range, the reference dimensions, and the positional relationship of the imaging object, which are equipped to the work vehicle of the present invention; [Diagram 2] FIG. 2 is a diagram showing the arrangement positions of the left and right imaging devices mounted on the work vehicle of the present invention, the reference dimensions, and the positional relationship of the front and left and right imaging objects; [Diagram 3] FIG. 1 shows a configuration for distinguishing between uncultivated and cultivated areas using an imaging device according to the present invention. [Figure 4] FIG. 1 shows a configuration for measuring the spacing between successively planted seedlings using an imaging device according to the present invention. [Diagram 5] Flow diagram of the present invention for determining a travel route based on the position of the crop [Figure 6] A configuration of a mobile terminal equipped with the touch screen of the present invention. [Figure 7] Flow diagram of changing steering direction with the touch screen of the present invention [Figure 8] A diagram showing the configuration of a disaster relief vehicle using the imaging device of the present invention. [Figure 9] FIG. 1 is a block diagram showing a configuration of an automatic driving system for a work vehicle according to the present invention; [Figure 10] FIG. 1 is a side view of a work vehicle according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] The present invention will now be described with reference to the embodiments shown in the drawings.

[0017] The work vehicle shown in Figs. 1 to 10 shows an example of this embodiment.

[0018] The work vehicle of the present invention will now be described.

[0019] Work vehicles include agricultural machinery and construction machinery, with a representative work vehicle being the tractor.

[0020] The main functions of the work vehicle 10 of the present invention will be described assuming that it is a tractor.

[0021] 9 is a block diagram showing the configuration of an automatic driving system A according to an embodiment of the present invention. First, the overall configuration of the automatic driving system A will be described below.

[0022] As shown in Fig. 9, the autonomous driving system A includes a work vehicle 10, an information terminal T1, and a management terminal T2. The work vehicle 1, the information terminal T1, and the management terminal T2 can communicate with each other via a communication network N using a communication device (not shown). As a communication means via the communication network N, for example, a mobile phone network, a packet line network, a wireless LAN, etc. can be used. In addition, the autonomous driving system A can acquire various data present on the cloud CL via the communication network N.

[0023] The work vehicle 10 is equipped with a work implement WM and is configured to be capable of automatically traveling in a field while performing agricultural work (hereinafter, sometimes simply referred to as "work"). As shown in FIG. 1, the work vehicle 10 is equipped with a travel control device C1 that controls the travel of the vehicle body and a work implement control device C2 that controls the work implement WM as control mechanisms. These control devices are information processing devices configured by combining multiple ECUs (Electronic Control Units). Each of the multiple ECUs is configured with a CPU that performs arithmetic processing and a memory that can read and write information required for the arithmetic processing, and the configuration shown as functional blocks in FIG. 9 is realized by the CPU operating in accordance with various control programs stored in the memory.

[0024] Furthermore, on the input side of these control devices, there are provided an obstacle detection sensor S1, a satellite positioning device S3, and an inertial positioning device S4.

[0025] The information terminal T1 is an information processing device that executes processing required for work in cooperation with the work vehicle 10, and transmits and receives various information required for work between the work vehicle 10 and the management terminal T2, thereby, for example, causing the work vehicle 10 to execute work and causing the management terminal T2 to manage and display various information related to the work. This information terminal T1 is configured, for example, by a device such as a personal computer, smartphone, or tablet terminal.

[0026] The information terminal T1 includes an obstacle classification processing unit t11, an obstacle information storage unit t12, and an operation unit t13. The operation unit t13 is an input device that receives various operations from an operator, and is configured by devices such as a keyboard and a touch panel, for example.

[0027] The management terminal T1 is an information processing device that mainly handles management functions, and transmits and receives various information required for work between the work vehicle 1 and the information terminal T1, thereby managing and displaying information related to work. This management terminal T1 is configured, for example, by a device such as a personal computer, smartphone, or tablet terminal.

[0028] Fig. 10 is a side view of a work vehicle 10 according to an embodiment of the present invention. In this specification, unless otherwise specified, the side in the traveling direction of the work vehicle 10 indicated by the arrow F in Fig. 10 is referred to as the front, the left side in the traveling direction of the work vehicle 10 is referred to as the "left", and the opposite side is referred to as the "right".

[0029] The work vehicle 10 in this embodiment is configured as a tractor, but is not limited to this and may be a rice transplanter, a combine harvester, a pest control machine, a harvester, or the like.

[0030] The work vehicle 10 is equipped with a traveling device M that drives the machine body. The traveling device M is equipped with an engine m1, front wheels m2, rear wheels m3, a transmission m4, a front axle m5, a rear axle m6, a steering wheel m7, etc., and is configured so that the steering wheel m7 can be automatically steered by the control of the traveling control device C1.

[0031] A work implement WM for performing work on the field R is attached to the rear of the work vehicle 10. The work implement WM is drive-controlled by a work implement control device C2. The work implement WM may be, for example, a cultivator, a grass cutter, a plow, a fertilizer applicator, or a seed sower, and is detachable from the main body of the work vehicle 1. In this embodiment, a cultivator is attached.

[0032] The obstacle detection sensor S1 is an ultrasonic sensor, and serves to detect obstacles around the work vehicle 10. This obstacle detection sensor S1 is equipped with a front obstacle sensor s11 that detects obstacles in front of the work vehicle 10, and side obstacle sensors s12, s12 that detect obstacles on the sides of the work vehicle 10.

[0033] The imaging devices 100 and 150 photograph the surroundings of the work vehicle 10 and acquire digital image data. Since the left and right imaging devices are the main function of the present invention, they are not shown in Fig. 10, and Fig. 1 shows the relationship between the imaging ranges and indicates the relationship between the equipment positions. In order to clarify the equipment positions, Fig. 1 omits the satellite positioning device S3, the inertial positioning device S4, the front obstacle sensor s11, the side obstacle sensors s12, s12 that detect obstacles on the sides of the work vehicle 1, and the optical imaging device S5.

[0034] The satellite positioning device S3 has a receiving antenna that receives radio waves from GNSS satellites, and is capable of acquiring position information of the current position of the work vehicle 10 based on the radio waves received by the receiving antenna.

[0035] The inertial positioning device S4 is a device (so-called IMU) that measures the three-axis tilt, acceleration, and orientation of the aircraft, and the inertial positioning device S4 makes it possible to calculate the traveling direction of the aircraft, correct the position information acquired by the positioning device S3, etc. In addition, for measuring the orientation, a separate orientation sensor may be provided.

[0036] The optical imaging device S5 is a 3D lidar, and obtains three-dimensional imaging information (3D image data) by irradiating an optical radar that emits light in a pulsed manner on a target and capturing scattered light. The obtained three-dimensional imaging information can be analyzed to detect the distance and shape of an object present in the irradiation range (detection range) of the optical radar. This makes it possible to obtain three-dimensional imaging information indicating the position and shape of an obstacle OB detected by the obstacle detection sensor S1. The optical imaging device S5 also includes a front optical imaging device s51 that detects an object in front of the work vehicle 1, and a rear optical imaging device s52 that detects an object behind the work vehicle 1. The irradiation range (detection range) of the optical imaging device S5 can be adjusted up, down, left, and right.

[0037] In the present invention, the optical imaging device S5 does not necessarily need to be equipped with a 3D lidar, and measurements can be made using the left and right imaging devices, which are the main function of the present invention. Therefore, by equipping both, more accurate measurements can be obtained, but for small plants such as seedlings that are aligned at intervals, as described below, the functions of the left and right imaging devices of the present invention are more effective.

[0038] The background of the present invention will now be described.

[0039] There is an increasing need for self-driving work vehicles in agricultural and construction work. In this type of self-driving, the vehicle travels along a route using a satellite positioning device or an inertial positioning device. There is technology that is equipped with an obstacle detection sensor and can make a detour or emergency stop if an object large enough to obstruct travel is detected, but crop seedlings are small in size and the obstacle sensor cannot detect the difference between them and weeds. Therefore, in self-driving, the vehicle may pass through a location where crops are planted. To deal with this, there is technology that provides an imaging device for identification, but the identification of crop seedlings and weeds is insufficient.

[0040] In this invention, multiple imaging devices are installed and images are taken at different angles while also capturing the reference position of the work vehicle in the image. By calculating three-dimensional dimensions from the work vehicle located in the field, it is possible to distinguish between crop seedlings and weeds, and between cultivated and uncultivated land.

[0041] This technology makes it possible to detect straight lines not only in front of the vehicle, but also in areas to the left and right, making it possible to make corrections to autonomous driving.

[0042] Referring to FIG.

[0043] The figure shows a left side view and a top view of a work vehicle 10 in a farm field, a person 95, and crops represented by plants 203 and 204 in the farm field.

[0044] The left imaging device 100 is provided at the left end 110 at the front of the roof part above the frame structure that protects the operation unit of the work vehicle 10, and the right imaging device 150 is provided at the right end 160. This operation unit is located inside a box-shaped room that is generally called a cabin in agricultural machinery, and is highly rigid due to its frame structure. In addition, the left end 110 and right end 160 at the front of the roof part are located at the upper limit of the height Z of the work vehicle, making it easy to check the distant direction. This rigidity allows the left imaging device 100 and right imaging device 150 to be stably fixed.

[0045] The advantages of performing wide-angle imaging using a plurality of imaging devices will now be described.

[0046] Imaging devices that detect 360 degrees around a work vehicle, such as LIDAR devices, take images continuously while rotating, so it is not easy to match the continuity of images when the work vehicle is moving fast or when it sways up and down. Also, when images are converted into point cloud data or formed into polygon shapes, it is difficult to measure the degree of change in the object shape itself by distance.

[0047] The present invention has the advantage of being able to provide immediacy and continuous image correspondence by fixing the imaging direction and range, allowing conditions and objects that can be constantly compared to be captured simultaneously by the left and right imaging devices, thereby enabling immediate distance measurement correspondence with a single shot of the image.

[0048] First, regarding the use of a stereo system, the left imaging central axis 101 of the left imaging device 100 is attached at an inclination of α1 to the left with respect to the travel axis 90 of the work vehicle. Therefore, by making use of the wide-angle imaging range of the left imaging device 100, it is possible to image the range of the left imaging wide angle 102. This range is characterized in that part of the front end position 180 of the hood of the work vehicle 10 and part of the rearview mirror 190 are included in the captured image within the imaging range.

[0049] Similarly, the left imaging center axis 151 of the right imaging device 150 is attached at an inclination of α2 to the left with respect to the travel axis 90 of the work vehicle. Therefore, by making use of the wide-angle imaging range of the left imaging device 150, it is possible to image the range of the right imaging wide angle 152. This range is characterized in that part of the front end position 180 of the hood of the work vehicle 10 and part of the rearview mirror 191 are included in the captured image within the imaging range.

[0050] In addition, in the vertical direction, the left imaging central axis 101 and the right imaging central axis 151 of the left imaging device 100 are inclined downward by β1 and the right imaging device 150 are inclined downward by β2 with respect to the travel axis 90 of the work vehicle.

[0051] Calculations are easier if the inclinations of each imaging device, α1 and α2, are the same angle, and β1 and β2 are the same angle, but if the work vehicle is always fixed in a constant direction, either clockwise or counterclockwise, there is also a method of shifting the angles to take turning into account.

[0052] By not aligning the central imaging axis of the imaging devices in the same direction as the axis of travel in this way, the imaging angles of the object for each imaging device differ, making it possible to calculate the object shape, distance, and each dimension in three dimensions.

[0053] A technique for simultaneously capturing an image of a reference object and determining the reference dimensions will be described.

[0054] The reason for providing multiple imaging devices, the left imaging device 100 and the right imaging device 150, is that it is possible to calculate three-dimensional distance by capturing images of the same object with two cameras with different image capturing positions. However, if the vehicle is on smooth asphalt, such as an automobile, it is unlikely to tilt forward, backward, left, or right, so there is almost no measurement deviation and measurements can be made with high accuracy. However, since the work vehicle 10 of the present invention travels in a farm field and tilts forward, backward, left, and right, the distance calculation may be deviated due to the scale of the captured image or the tilt of the imaging direction due to focus deviation in the imaging device, slight time lag in shooting, and difference in light amount, and therefore it is necessary to carry out image correction relative to the absolute numerical scale.

[0055] This relative image correction is achieved by a method that easily incorporates a portion of the work vehicle into the captured image. In incorporating this portion, the front portion includes the front end position 180 of the hood of the work vehicle 10, and portions of the left rearview mirror 190 and right rearview mirror 191 on the left and right sides are incorporated into the captured image.

[0056] By incorporating this part, the angle, distance, and tilt shifts in the imaging direction of the left and right imaging devices are corrected, and by combining the data from the left and right imaging devices, corrections are made when calculating three-dimensional distance.

[0057] Even if the performance of each imaging device is high and imaging is possible at a wide angle of more than 180 degrees, the front end position 180 of the hood of the work vehicle and each rearview mirror are imaged, making it easy to correct the scale using the reference. In this way, by capturing images of fixed objects that serve as references in front and on the sides on the left and right imaging devices, respectively, the distance to the target object can be measured not only in absolute terms but also by performing relative correction to calculate a more accurate distance. In particular, it was difficult to measure the positional relationship of plants that are planted continuously at equal intervals, such as seedlings in a field, but by measuring the three-dimensional distance as in the present invention, the three-dimensional thickness of the object is read by a stereo camera, and the interval between each seedling can be calculated, and by using the fixed objects in front and on the sides in the image as a reference, it is possible to calculate the line 200 on the side in FIG. 3.

[0058] The imaging of this fixed object will now be described. In this embodiment, it is necessary to image the front end position 180 of the hood of the work vehicle. To achieve this, a position that is rearward of the front end position 180 of the hood in the traveling direction of the vehicle, that is, an intersection 170 of the right end line 103 of the wide angle 102 of the imaging of the left imaging device 100 and the left end line 153 of the wide angle 152 of the imaging of the right imaging device 150, is configured to be rearward of the front end position 180 of the hood of the vehicle. This measurement, together with the reference dimension 181 of the hood length and the reference dimension 186 of the hood height that have been measured in advance, make it possible to correct the measurement of the position of the object in front.

[0059] Furthermore, by including a part of the left rear-view mirror 190 and the right rear-view mirror 191 within the image capture screen, the position of the rear-view mirror is measured, and a reference dimension 184 of the left rear-view mirror height and a reference dimension 185 of the right rear-view mirror height that have been measured in advance are obtained, and a left rear-view mirror position 182 and a right rear-view mirror position 183 are also registered.

[0060] The left rearview mirror 190 and the right rearview mirror 191 are attached at left and right positions with a distance wider than the width dimension X of the front and rear wheels of the work vehicle 10. This is because when a person is riding and driving, in order to check behind, they cannot see anything unless it is outside the width dimension X of the front and rear wheels. Therefore, by arranging the left imaging device 100 and the right imaging device 150 at the left end 110 and right end 160 in front of the roof part of the frame of the operation unit, the mounting positions of the important imaging devices are located inside the width dimension X of the front and rear wheels, eliminating contact with obstacles etc. while driving and dealing with damage.

[0061] In addition, placement at the left end 110 and right end 160 in front of the roof part of the frame of the operating part means that it can be located either outside or inside the cabin called the operating part, and is fixed near the frame at the corner, and can also be placed on the windshield part if it does not affect the opening and closing of the door.

[0062] That is, the left imaging device 100 and the right imaging device 150 may be mounted inside the operation section, that is, inside the cabin. The advantage of this position is that the imaging devices are protected from natural elements such as dust, rain, and wind. It is possible to provide the imaging device itself with water resistance and dust resistance, but if dust adheres to the lens section, it is difficult to remove it during automatic driving. However, if it is mounted inside the cabin, automatic removal is possible by providing a wiper to remove dust from the cabin glass.

[0063] The left and right imaging devices are protected because they are disposed as described above, and the left and right rearview mirrors located nearby are not considered to be blind spots of the imaging devices, but rather, a method is used to register the reference dimensions on the image. Since the positional relationship of the rearview mirror is changed by the user who operates it, the rearview mirror position, which is the reference dimension, is changed and registered at the beginning of the operation, and the reference dimensions are also changed. For example, if the rearview mirror is electrically operated, the arrangement position is automatically registered, and the reference dimensions 182, 183, 184, and 185 are also changed, and the change data is used for image analysis.

[0064] The relationship of the reference dimensions to be captured in such an image is characterized in that three-dimensional positions are registered in three directions, the front, right, and left, and this is the basic technology of the technology of the first invention. In the embodiment, the reference dimensions are described as three-dimensional positions in three directions, but it is also possible to calculate by using the reference dimensions only for the front in the calculation of the left and right sides. In this case, if there is a large tilt to the left or right, correction can be made if an inertial positioning device is used, but normally the image data is processed to be excluded.

[0065] This relative distance correction makes it possible to detect the difference in the positions of the seedlings 203 and 204 in the field. If the image is detected directly as in the conventional technology, the boundary between the seedlings and the line 201 of the ground position is unclear because the seedlings 203 and 204 overlap, and the boundary is unclear, so the separation of the seedlings is unclear. Since it is a stereo camera, there is a difference between the images of the left and right imaging devices, but the further away the difference is, the smaller the difference becomes and the more unclear it becomes. Even with such images, it is possible to correct them using a technology that performs a relative ratio calculation of the reference dimensions and moving speed of a fixed object in three dimensions and the image change. In other words, it becomes possible to calculate the arrangement of objects and objects that cannot be measured unless they are photographed from the sky by a drone, etc.

[0066] 2 shows the measurement of the distance L1 to the forward position A1, the distance L2 to the left side A2, and the distance L3 to the right side A3. It may be considered that the distances are measured in a ripple pattern from the left and right imaging devices. The distance L1 to the forward position A1 can be calculated by utilizing the effects of both the range of the wide angle 102 of the imaging of the left imaging device 100 and the range of the wide angle 152 of the imaging of the right imaging device 150. The range of the wide angle 102 of the imaging and the range of the wide angle 152 of the imaging can be further expanded depending on the performance and mounting position of the imaging device, and it is possible to expand the range using the front stereo camera.

[0067] In Fig. 2, A2 on the left side and A3 on the right side indicate areas that can only be imaged by an imaging device on one side. In this case, the basic dimensions 182 and 183 to the rearview mirror mentioned above are the main focus. By performing a ratio calculation and correcting the directly measured distance based on a pre-registered image standard, it becomes possible to measure the distance L2 to A2 on the left side and the distance L3 to A3 on the right side. As a result, A2 on the left side is shown as distances LE1 and LE2, as distances that match the traveling direction of the work vehicle. Also, A3 on the right side is shown as distances R1 and R2.

[0068] 3 explains the second invention. Based on the difference in the images of the soil of the farmland on each side of the left image capturing device 100 and the right image capturing device 150, the positions of the uncultivated and cultivated areas are determined, and a straight line for automatic travel is determined based on a line 200 in the cultivated area.

[0069] The left and right image capture devices can also determine color shades, detecting the degree of the three primary colors, RGB. This technology makes it possible to use the image capture devices to distinguish between uncultivated and cultivated areas during cultivation work.

[0070] The right imaging device 150 detects P1, which is the shortest distance to the uncultivated land threshold. The composite image of the right imaging device 150 and the left imaging device 100 also captures an image of the front end position 180 of the vehicle's hood, and P2, the uncultivated land threshold, is detected.

[0071] The distance L9 is calculated based on P1 and P2, and at the same time the line 200 of the already tilled part is determined, and automatic driving is performed based on the line 200 of the already tilled part. Thereafter, the vehicle drives so that the distance L7, which is the center dimension of the work vehicle calculated by the imaging device, matches the travel axis 90 of the work vehicle. Since matching the measured distance L5 and the registered distance L6 will result in errors, priority is given to control so that the distance L7 matches the travel axis 90 of the work vehicle.

[0072] Also, the imaging angle α3 of the left imaging device 100, once measured and determined, may be fixed, and the distance L4 of P2 may be adjusted.

[0073] 4 explains the third invention. As described above, the left imaging device 100 and the right imaging device 150 can detect color shades. This makes it possible to detect the position of green, recognize continuously connected lines on the image as planting lines for crops, and match them with the width dimension (X) of the front and rear wheels, and automatically travel so that the front and rear wheels do not overlap the planting line position.

[0074] Conventional technology uses a satellite positioning device to drive along a line. However, this technology mainly involves driving in a straight line, so there are cases where the wheels are aligned with areas where crops are planted. The feature of the present invention is a technology that recognizes continuously connected lines as crop planting lines through image analysis by an imaging device. If greenery is simply recognized as plants, weeds will be erroneously detected. Therefore, it is necessary to identify planted crops that are planted on the same line and at the same intervals.

[0075] As explained above, in the present invention, a technique for detecting the interval 205 between seedlings 203 and seedlings 204 is used to detect the line 201 between the seedlings and their ground positions. In farm fields, weeds grow around seedlings. For this reason, it may be difficult to determine whether a seedling is a weed or a seedling using only an imaging device. Therefore, in the present invention, the seedlings are planted at equal intervals, in a straight line or in a staggered pattern, and are not planted in an irregular manner at all, so that line detection is performed using the technique of the present invention that can detect the planting intervals.

[0076] This technology makes it possible to correctly detect lateral lines such as line 208 and line 203, thereby preventing automatic driving that would result in the wheels of a work vehicle passing directly above these lines.

[0077] Furthermore, line detection technology is used to grasp the positional relationship between the width dimension X of the front and rear wheels and the positions of lines 208, 201, and 202. The right side of the work vehicle has a distance X5, which is the difference from the center position of the work vehicle, and therefore the left side has a distance of X8 and X5. In addition, since work vehicles have wheel widths, the innermost dimensions of the front and rear wheels are necessary. In this embodiment, the right side has a distance of X9, and the left side has a distance of X10, and it is detected that problem-free driving is possible.

[0078] A technique for detecting the interval 205 between seedlings 203 and 204 will be described with reference to Fig. 4. For seedling 203, the right imaging device 150 is located at a position where a wide angle of α8 is extended to the left from the central axis, and for the left imaging device 100, it is located at a position where a wide angle of α7 is extended to the right from the central axis. For seedling 204, the right imaging device 150 is located at a position where a wide angle of α9 is extended to the left from the central axis, and for the left imaging device 100, it is located at a position where a wide angle of α6 is extended to the right from the central axis. By detecting the imaging angle and distance, it is possible to detect the interval 205 between seedlings 203 and 204 and the distance 206 from the work vehicle.

[0079] FIG. 5 is a flow diagram for determining a travel route based on the position of the crop.

[0080] Crop recognition S5-1 is performed using the left and right imaging devices, and the planting intervals of the crops are measured to recognize the planting lines S5-2. This refers to lines 208 and 201 in Figure 4. As mentioned above, the measurement technology can calculate the planting intervals S5-4 of the crops from the dimensions detected by the imaging devices, the work speed of the work vehicle, and the captured image data S5-3. If detection is performed in the horizontal direction perpendicular to this direction, it is also possible to recognize the ridges and row spacing.

[0081] By comparing the preset driving route S5-5 with S5-6, which is the width dimension X of the front and rear wheels, the route correction offset S5-7, which is X9 on the right side as shown in Figure 4, is calculated, allowing the vehicle to drive across the crops.

[0082] This section describes the configuration for driving an unmanned autonomous agricultural machine using gesture operations on a touch screen.

[0083] This technology is a remote control technology. Conventional operating devices require pressing buttons for forward, reverse, left steering, and right steering exclusively, making it difficult to operate simultaneously and steering in a smooth curve. However, when working in fields, etc., if a human is seated in the operating seat and driving, it is possible to perform efficient continuous work by driving forward and backward while changing the turning angle, and driving to match a curve.

[0084] When operating this with a remote control, since the operating device is a portable terminal, the operating switch section for steering and the operating switch section for forward and reverse driving are separate, so it is necessary to modify the operating section so that they can be easily operated simultaneously using the portable terminal.

[0085] In the present invention, steering is performed by utilizing the functions of the touch screen 252. The touch screen 252 is a capacitance type, and detects the touch position from the slight change in capacitance that occurs between the finger and the touch panel. By detecting this position, steering can be performed according to the curve drawn by the finger.

[0086] 6 includes a slider device 251 for moving forward and backward, and moving the bar up to position 253 increases the forward speed, and moving the bar down to position 254 increases the reverse speed. The touch screen 252 is rotatable and is linked to the steering movement.

[0087] If you make a path with your finger like right rotation 255, the steering will be to the right. If you make a path with your finger like left rotation 256, the steering will be to the left.

[0088] This finger movement is called a swipe operation, and it is possible to control not only the swipe direction but also the swipe speed. Figure 7 shows how it is linked to the steering operation speed. The faster you turn the steering wheel, the faster you can operate the steering wheel.

[0089] The touch screen recognizes a swipe operation on a circular operation section on the touch screen (S7-1). The swipe operation (S7-2) is detected by detecting the position of the finger, and two actions are detected by the position detection: the swipe direction (S7-3) in which the finger is pointing, and the swipe operation speed (S7-4) which is the speed at which the finger moves.

[0090] In the swipe direction S7-3, the steering direction is determined by the direction of rotation of the touch screen as described above. In the clockwise direction S7-5, the direction of travel is set to the right S7-6, and in the counterclockwise direction S7-7, the direction of travel is set to the left S7-8.

[0091] In the swipe operation speed S7-4, the steering operation speed S7-9 is determined from the swipe operation speed. The faster you turn your finger, the faster you can steer.

[0092] The direction of travel and speed of operation can be controlled by sending operation signals S7-10 from the touch screen, and the forward and reverse directions and speed can also be controlled on the touch screen terminal 250 by using slide switches as described above.

[0093] The operation and motion of the work vehicle can be selected, with rotation of the touch screen 252 resulting in forward and reverse movement of the work vehicle and swipe speed resulting in forward and reverse speed.

[0094] If more continuous driving and steering is desired, a driving route can be drawn in the center of the touch screen 252, and the vehicle will steer and drive according to that route.

[0095] The special vehicle for disaster relief shown in FIG. 8 will now be described.

[0096] The illustrated work vehicle 300 is equipped with an AI camera 310 in addition to the functions of the present invention, and is a robot work vehicle with the function of responding when a disaster occurs. The operator's seat and the center of the main body are equipped with a solar panel 320. The size of the solar panel 320 is also large enough, and its area is equivalent to the size of a bed 330 on which a person can lie down. With this configuration, in disaster relief, the bed 330 can be temporarily placed on the solar panel 320 to respond to rescue lives.

[0097] We will now explain the AI ​​camera 310. This camera is a camera with artificial intelligence functions. It can detect people, specific objects, and if there is a certain pattern in the captured image, it can recognize that it is the same pattern if it detects that phenomenon. It is also possible to recognize specific colors and areas from colors.

[0098] In this application, an image of a person lying down is used as a reference image and the data is stored in the CPU of the work vehicle 300 or in the cloud, and by comparing the shape with this image, it is possible to determine that it is a person lying down.

[0099] Generally, a person lying down is likely to have hurt their leg, and may be able to move their arm. In a person's pattern behavior, if some kind of sound or voice call is made from the work vehicle 300, the person may react by moving their arm, and the timing of this sound and the person's pattern behavior are registered as a comprehensive pattern, and by comparing the image pattern, the function of judging it as a person is provided.

[0100] In addition, the captured image is analyzed using RGB color analysis to detect whether there are any colors other than skin tones in the face. If red to brown colors are detected, it can be determined that there is a possibility of bleeding. If the color is dark brown to black, it is determined that there is mud or dust on the face, and it is presumed that there is a significant injury.

[0101] In the embodiment described here, a dedicated disaster vehicle is used, but even machinery used as normal agricultural machinery can be equipped with an AI camera 310 and used as a disaster vehicle.

[0102] As for the solar panel 320, if it is larger and has a wider surface area, a part of it can be folded up and used as a windbreak. Also, if it is folded up on an incline, it can function as a tarp to temporarily protect against rain and wind.

[0103] This function can be implemented as an additional function by utilizing a work vehicle such as an agricultural machine or a construction machine, and does not have to be dependent on a specific work vehicle. [Explanation of symbols]

[0104] 10 Work vehicles 100 Right imaging device 150 Left imaging device 170 Intersection of right edge line 103 and left edge line 153 180 Front end position of vehicle hood 190 Rearview Mirror

Claims

1. The vehicle is provided with a left imaging device (100) at a left end (110) at the front of a roof part above a frame structure that protects an operation part, and a right imaging device (150) at a right end (160), The central axis (101) of the photographing direction of the left imaging device (100) is inclined at an angle (α1) to the left and an angle (β1) downward from the traveling axis (90) of the vehicle body, and the center of focus is located below the left side of the vehicle body. The central axis (151) of the photographing direction of the right imaging device (150) is inclined at an angle (α2) to the right and an angle (β2) downward from the traveling axis (90) of the vehicle body, and the center of focus is located below the right side of the vehicle body. A left imaging device (100) and a right imaging device (150) are disposed within the width dimension (X) of the front and rear wheels in the width direction of the vehicle, an intersection (170) of a right end line (103) of the wide angle (102) of the imaging of the left imaging device (100) and a left end line (153) of the wide angle (152) of the imaging of the right imaging device (150) is located behind a front end position (180) of a hood of the vehicle; The left imaging device (100) and the right imaging device (150) capture images of the front end position (180) of the vehicle hood, and by calculating the ratio of the reference dimension (181), A work vehicle that measures the distance (L1) to a forward position (A1), the distance (L2) to a position on the left side (A2), and the distance (L3) to a position on the right side (A3).

2. The work vehicle of claim 1, which determines the positions of the uncultivated and cultivated areas based on the difference in images of the soil areas of the field on each side of the left and right imaging devices (100, 150), and automatically travels along a straight line based on the line (200) of the cultivated area.

3. The work vehicle according to claim 1 or 2, wherein the left imaging device (100) and the right imaging device (150) detect the position of greenery, recognize continuously connected lines on the images as crop planting lines, compare them with the width dimension (X) of the front and rear wheels, and automatically travel along the planting line position so that the positions of the front and rear wheels do not overlap.

Images