Obstacle Monitoring System

Abstract

To provide a safety confirmation-type obstacle monitoring-system responsive to information about obstacles that exist in a farm field and may lead to an accident, the system detecting obstacles, avoiding abnormality such as stopping a working vehicle, continuing work according to a situation, or the like.SOLUTION: When a work field sensor 500 confirms the entry of an obstacle into a work field A and an obstacle sensor 130 of a work vehicle 10 detects the obstacle, a control device 135 mounted on the work vehicle 10 stops the work vehicle 10; when the work field sensor 500 confirms the entry of an obstacle into the work site A, but the obstacle sensor 130 of the work vehicle 10 does not detect the obstacle, the vehicle speed of the work vehicle 10 is reduced, and a cloud server 700 transmits a notification of a work route change to a terminal 140.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an obstacle monitoring system for managing operations of various work vehicles such as agricultural machines and civil engineering machines. [Background technology]

[0002] There is an agricultural work support system that makes it possible to acquire work progress information such as information on work not yet started, information on work in progress, and information on work completion for each field, and that makes it possible to update the work progress information during work (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-064663 Summary of the Invention [Problem to be solved by the invention]

[0004] Such conventional agricultural work support systems acquire general information related to the progress of work and the status of the work equipment, and update the work information, but do not respond to information about obstacles such as work equipment, workers, or animals that exist in the field.

[0005] The present invention has been made in consideration of the above, and aims to provide an obstacle monitoring system that responds to information about obstacles that may lead to accidents within a farm field, and that not only detects obstacles and avoids abnormalities by stopping the work vehicle, but also responds to safety confirmation, such as by continuing work depending on the situation. [Means for solving the problem]

[0006] The invention described in claim 1 is an obstacle monitoring system including an obstacle sensor 130 and a control device 135 mounted on a work vehicle 10, an external mobile terminal 140, a work site sensor 500 for detecting intrusion into and exit from work sites A, B, and C, and a cloud server 700, The work site sensor 500 monitors the intrusion and exit of an obstacle 800 into the work sites A, B, and C, and when it detects the intrusion of an obstacle 800, it transmits the intrusion information to the cloud server 700. The obstacle sensor 130 of the work vehicle 10 transmits information on the presence or absence of an obstacle 800 from the control device 135 mounted on the work vehicle 10 to the external mobile terminal 140 and the cloud server 700, When the work site sensor 500 confirms the intrusion of an obstacle 800 into work sites A, B, C, and the obstacle sensor 130 of the work vehicle 10 detects the obstacle 800, the control device 135 mounted on the work vehicle 10 stops the work vehicle 10 and transmits a work process stop to the cloud server 700, and the cloud server 700 transmits a work process stop to the terminal 140, If the work site sensor 500 confirms the intrusion of an obstacle 800 into work sites A, B, or C, but the obstacle sensor 130 of the work vehicle 10 does not detect the obstacle 800, the obstacle monitoring system reduces the speed of the work vehicle 10, changes the work process time based on the reduction in vehicle speed, and sends a notification of the work process change to the cloud server 700, and the cloud server 700 sends a notification of the work process change to the terminal 140.

[0007] According to the invention described in claim 1, information on obstacles 800 existing in the work sites A, B, and C can be integrated into the cloud server 700 and used to modify the work plan on the fly.

[0008] If an obstacle 800 enters work areas A, B, or C, it is a condition requiring attention and is immediately reported to the worker (supervisor), but if the obstacle sensor 130 of the work vehicle 10 does not detect it, there is no obstacle 800 in the vicinity of the work vehicle 10, so the work speed is reduced to avoid significant delays in the work. By correctly and instantly analyzing the information on the obstacle 800 in this way, accidents can be prevented while also correcting the work plan, making driving the autonomous work vehicle 10 close to the level of manned driving.

[0009] The invention described in claim 2 is characterized in that, after the work site sensor 500 confirms the intrusion of an obstacle 800 into work sites A, B, and C, it confirms that the obstacle 800 has left the work sites A, B, and C, and when the obstacle sensor 130 of the work vehicle 10 goes from detecting the obstacle 800 to not detecting it, the operation of the stopped work vehicle 10 is started, the work speed is set to a set speed, the work process time is changed based on the time during which work was stopped, a notification of the change in the work process is sent to the cloud server 700, and the cloud server 700 sends a notification of the change in the work process to the terminal 140, This is an obstacle monitoring system as described in claim 1, in which after the work site sensor 500 confirms the intrusion of an obstacle 800 into work sites A, B, C, it confirms that the obstacle 800 has left work sites A, B, C, and if the obstacle sensor 130 of the work vehicle 10 does not detect the obstacle 800, the reduced work speed is returned to the set work speed, the work process time is changed, and a notification of the change in work process is sent to the cloud server 700, and the cloud server 700 sends a notification of the change in work process to the terminal 140.

[0010] According to the invention described in claim 2, even if the obstacle 800 is an animal, it is unlikely that it will stay in the field for a long time, and once it is confirmed that the animal has left the work areas A, B, and C, work can be resumed, or if the work speed has been reduced, the work speed can be returned to normal, allowing for the same response as with manned monitoring.

[0011] The invention described in claim 3 is an obstacle monitoring system described in claim 1, in which if the obstacle sensor 130 of the work vehicle 10 detects an obstacle 800 and suddenly loses information within the detection range of the obstacle sensor 130, if the size of the obstacle 800 exceeds a specified dimension, or if the obstacle 800 is approaching the work vehicle 10 at a relative speed, the work vehicle 10 is stopped and moving obstacle abnormality information is transmitted to the cloud server 700, and the cloud server 700 transmits the moving obstacle abnormality information to the terminal 140.

[0012] According to the invention described in claim 3, it is possible to respond when it is determined that the obstacle 800 is a large animal or a human. For example, any of the following cases may lead to an accident: an accident involving an obstacle, an accident resulting in injury to a person, a theft, a property damage accident, a prank, or a hit-and-run. When the vehicle stops, information at that time is sent out, and if it turns into a serious accident, it can be saved as final data.

[0013] The invention described in claim 4 is an obstacle monitoring system described in any one of claims 1 to 3, in which an FMIS (Farm Management Information System, hereinafter referred to as FMIS) is a system that centrally manages data detected by each sensor of a work vehicle 10, data processed by a personal computer 610 by adding work data and farm management information data accumulated from past work from a cloud server 700, and the data processed by the personal computer 610. The cloud server 700 communicates with each terminal 135, 140, 610, and the cloud server 700 instantly compares FMIS data, manages the position information of the work vehicle 10, an obstacle 800, and each work site A, B, C, performs mapping processing, and transmits the mapping processing information to each terminal 135, 140, 610.

[0014] According to the invention as recited in claim 4, by managing the position information of the work vehicle 10 and obstacles 800 using a mapping diagram, this can be expanded to information management of work at other work sites A, B, and C.

[0015] There is also a possibility that this could lead to crimes or accidents, so by instantly obtaining sensor data such as fuel, oil temperature, battery level, speed, gear range, wheel rotation speed of the work vehicle 10, and obstacle detection from the work vehicle 10, or by comparing this with accumulated data from the external mobile terminal 140, the data can be used as analysis data in the event of an accident, etc.

[0016] In addition, by enabling communication via FMIS, a system that centrally manages existing processed data and real-time detection data, obstacles 800 can be instantly integrated into existing mapping data, making it possible to remotely respond to changes in work processes (work times) in real time even with an autonomous work vehicle 10.

[0017] In addition, when there are workers at work sites A, B, and C, the integrated mapping data can be viewed on the external mobile terminal 140, so that any changes to the work process can be checked with the cloud server 700 and the changes to the work process can be immediately confirmed on-site. [Brief description of the drawings]

[0018] [Figure 1] 1 is a configuration diagram showing an outline of an obstacle monitoring system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a block diagram showing an outline of the obstacle monitoring system. [Diagram 3] FIG. 2 is a block diagram showing an outline of the obstacle monitoring system. [Figure 4] 4A and 4B are plan views for explaining the operation of the above system in a farm field. [Diagram 5] FIG. [Figure 6] 4A and 4B are diagrams showing a farm field-related information table and a work-related information table of the obstacle monitoring system. [Figure 7] FIG. 2 is a block diagram of a variable operation map of the system. [Figure 8] This is the map data display screen of the above system. [Figure 9] FIG. 2 is a control flow diagram of the obstacle monitoring system. [Figure 10] FIG. 2 is a control flow diagram of the obstacle monitoring system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, with reference to the drawings, a specific description will be given of an obstacle monitoring system as a preferred embodiment of the present invention when a tractor 1 equipped with a rotary tiller 2 as a working machine performs tilling work in fields A, B, and C as working areas. Note that the components in the following embodiment include those that are replaceable and easy for a person skilled in the art, or those that are substantially the same, that is, those within the scope of what is called equivalents. Furthermore, the present invention is not limited to the embodiment, and various modifications can be made without departing from the gist of the present invention.

[0020] 1 and 2 are schematic diagrams of an obstacle monitoring system, and FIG. 3 is a block diagram showing an outline of the obstacle monitoring system.

[0021] The obstacle monitoring system includes, for example, a work vehicle 10, a GNSS control device 120 as a position information acquisition device that acquires position information indicating the position of the work vehicle 10, an obstacle sensor 130 mounted on the work vehicle 10, a control device 135 mounted on the work vehicle 10, an external mobile terminal 140 such as a tablet terminal as an information processing device that can be carried onto the work vehicle 10, a fixed sensor 500 in the field as a work field sensor that detects entry into and exit from fields A, B and C (leaving fields A, B and C), a personal computer 610 as an information processing terminal provided in a ground base 600, and a cloud server 700.

[0022] The GNSS control device 120 is equipped with a receiving antenna 122 provided on the traveling vehicle body 1 and a GNSS antenna 121 provided on the external mobile terminal 140, and receives radio waves from a positioning satellite 123 using the GNSS antenna 121 and the receiving antenna 122, and acquires GNSS coordinates at predetermined time intervals, thereby obtaining position information on the earth at predetermined intervals.

[0023] Then, a table is generated in which the location information acquired by the GNSS control device 120, the field-related information for the fields A, B, and C, and the work-related information for each location are recorded in association with field map information indicating the locations of the fields A, B, and C, as independent information for each of the fields A, B, and C.

[0024] The obstacle sensor 130 is a sensor for detecting general obstacles and is provided at the front, both left and right sides and rear of the work vehicle 10. It detects obstacles in all directions around the work vehicle 10 and outputs obstacle detection data to the control device 135.

[0025] The control device 135 mounted on the work vehicle 10 is provided with a processing device having a CPU and the like, storage devices such as a ROM, a RAM, and a HDD, a monitor 136, and input / output devices.

[0026] In addition, the control device 135 is connected to an ECU, which is an engine control device, an automatic driving selection switch, various sensors, an electric motor, various actuators such as a hydraulic cylinder, and the like, as well as to the GNSS control device 120 and an automatic steering device.

[0027] The various sensors connected to the control device 135 include, for example, a water depth sensor that detects the water depth in fields A, B, and C, a cultivated soil depth sensor that detects the cultivated soil depth in fields A, B, and C, a fertility sensor that detects the fertilizer concentration in fields A, B, and C, weight sensors such as load cells that detect the weight of harvested crops such as rice or the weight of seedlings, a rotation sensor that detects the rotation speed of the rear wheels, an inclination sensor that detects the inclination of the traveling body 1, a work clutch sensor, a temperature sensor, and various other sensors.

[0028] The various actuators include various cylinders, such as a lifting cylinder that raises and lowers the working device 2, motors that open and close the water gates installed at the water inlets that serve as water supply and drainage points to the fields A, B, and C, motors that rotate water depth sensors, and various motors, such as a throttle motor that adjusts the amount of air intake to the engine.

[0029] The automatic steering device is controlled by the control device 135 based on position information acquired by the GNSS control device 120, and can automatically operate a steering wheel provided on the traveling vehicle body 1 to automatically drive the traveling vehicle body 1. The automatic steering device includes a steering motor that applies an arbitrary rotational force to rotate the steering wheel, and a steering wheel potentiometer that detects the rotation angle of the steering wheel.

[0030] The external mobile terminal 140 communicates data with the cloud server 700 using the FMIS via a communication network.

[0031] The in-field fixed sensors 500 are monitoring equipment using imaging means installed at diagonal positions in each of the fields A, B, and C. As shown in Figures 4(A) and (B), taking field A as a representative example, each sensor monitors the direction of the field boundary line AL at a diagonal position in field A, thereby detecting the intrusion and exit (exit from field A) of obstacles 800 such as animals, people, and other work machines into field A. The images captured by the imaging means are subjected to image analysis using dots to detect the intrusion and exit (exit from field A) of the obstacles 800 from field A.

[0032] Information on the entry of an obstacle 800 into or exit from fields A, B, and C detected by the fixed sensor 500 in the field is exchanged between the FMIS and the cloud server 700 in shp format or XML via a communication network.

[0033] The personal computer 610 is equipped with a processing device having a CPU etc., storage devices such as ROM, RAM, and HDD, a monitor 610a, and an input / output device, and communicates data with the cloud server 700 in shp format or XML using FMIS via a communication network.

[0034] 5 and 6, the storage device stores the field-related information, work-related information and work vehicle position information as independent information for each of the multiple fields A, B and C in association with field map information indicating the locations of the fields A, B and C, each of which has been assigned a field representative point A1, B1 and C1. Note that similar information is recorded for the fields D to I in Fig. 5.

[0035] As shown in Figure 6, the field-related information includes field area information and field representative point positions for each of the fields A, B, and C, and the work-related information includes information by work machine type and machine information for each of the fields A, B, and C.

[0036] Here, with reference to Figures 7 and 8, we will explain the transmission of information between the cloud server 700, the personal computer 610, the external mobile terminal 140, and the control device 135 installed in the work vehicle 10 regarding the variable work map of field-related information, which is the work data.

[0037] As shown in FIG. 7, a personal computer 610 communicates data in shp format or XML with a cloud server 700 in the FMIS via a communication network.

[0038] The cloud server 700 communicates data with the external mobile terminal 140 in shp format or XML via a communication network.

[0039] The external mobile terminal 140 communicates data with the control device 135 mounted on the work vehicle 10 in shp format or XML via Bluetooth (registered trademark).

[0040] The personal computer 610, the external mobile terminal 140, and the control device 135 mounted on the work vehicle 10 can also share information via USB.

[0041] Then, as shown in FIG. 8, the personal computer 610 displays the amount of fertilizer applied to the field B for the last year in five stages from set value 1 to set value 5 on the field map information (map data display).

[0042] The work vehicle 10 autonomously drives along the driving route information of the field-related information while calculating position information using the GNSS control device 120 in field B and automatically performs tilling work. The work vehicle 10 sends position information of point A, which is the starting point of the first process, and point B, which is the end point, to a personal computer 610 via a cloud server 700.

[0043] The personal computer 610 then compares the points A and B of the work vehicle 10 sent with the points A and B of the driving route information, and if they match, turns on the "Automatic driving permitted" displayed on the monitor 610a.

[0044] The worker (supervisor) confirms that the automatic driving possible light is on and that the field in which he is currently working is field B, so he presses OK.

[0045] Then, the field-related information and work-related information related to the field map information are integrated into a variable work map and displayed on the monitor 610a. When the worker (supervisor) confirms the information and presses OK, data is exchanged in shp format or XML from the personal computer 610 to the cloud server 700, the external mobile terminal 140, and the control device 135 mounted on the work vehicle 10, and the variable work map is shared.

[0046] Then, the external portable terminal 140 and the control device 135 mounted on the work vehicle 10 read the variable work map and display it on the monitors 142, 136 respectively, and the work vehicle 10 travels autonomously along the driving route information and automatically performs tilling work.

[0047] At this time, the worker adjusts the working conditions, such as the tilling depth, in accordance with the amount of fertilizer applied last year, while checking the position of his / her own vehicle on the variable work map displayed on the monitors 142, 136.

[0048] Although the above example shows the variable work map being integrated using the personal computer 610, the map may also be integrated using the external mobile terminal 140 or the control device 135 mounted on the work vehicle 10 and shared with other cloud servers 700, etc.

[0049] As an optimal example, the most efficient operation is performed when a worker aboard the work vehicle 10 looks at the monitor 136 on the control device 135 mounted on the work vehicle 10 and performs map linkage (operation in FIG. 8).

[0050] The FMIS is a system that adds work data and farm management information data accumulated from past work from the cloud server 700, processes the data using a personal computer 610, and centrally manages the data detected by each sensor of the work vehicle 10. The cloud server 700 can communicate with each terminal 135, 140, 610.

[0051] The cloud server 700 also instantly collates FMIS data, manages the position information of the work vehicle 10, obstacle 800, and each of the fields A, B, and C, performs mapping processing, and transmits the mapping processing information to each of the terminals 135, 140, and 610.

[0052] The work vehicle 10 is equipped with a travelling body 1 capable of travelling in the fields A, B, and C, and a working implement 2. For example, if the work vehicle 10 is a tractor 1, the working implement 2 is a rotary tilling implement 2, if the work vehicle 10 is a seedling transplanter, the working implement is a seedling planting implement, if the work vehicle is a fertilizer applicator, the working implement is a fertilizer applicator, etc. If the work vehicle 10 is, for example, a combine harvester, the working implement is a reaping section, a threshing section, etc. The above example is merely an example, and the working implement is not particularly limited as long as it is a working implement for performing agricultural work in the fields A, B, and C.

[0053] The traveling vehicle body 1 is equipped with an engine as a power source and a power transmission device that transmits the power of the engine to the drive wheels and the working implement 2, and can travel freely inside farm roads R and fields A, B, and C. The engine used may be a heat engine such as a diesel engine or a gasoline engine.

[0054] The external mobile terminal 140 is a type of computer similar in configuration to the personal computer 610 described above, and is equipped with a control unit, a memory unit connected to the control unit, a monitor 142 which is a touch panel that combines a display unit for displaying information and an operation unit for performing various input operations, and a GNSS antenna 121.

[0055] The memory unit stores field-related information, work-related information and work vehicle position information as independent information for each of the multiple fields A, B, and C, in association with field map information indicating the locations of the fields A, B, and C, which are each assigned field representative points A1, B1, and C1.

[0056] In the obstacle monitoring system with the above configuration, the field-related information, work-related information, and position information of the work vehicle 10 can be managed in a centralized manner for each of a plurality of fields A, B, and C that can be visually identified by map information. Therefore, the cloud server 700 can associate each piece of information with each other and display it on the screen of the monitor 142 of the external mobile terminal 140, for example, making it easy to check visually. Furthermore, the necessary information in which the field-related information, work-related information, and position information of the work vehicle 10 are associated with each other can be shared by a personal computer 610 located away from the fields A, B, and C, making it easy to plan agricultural work, etc.

[0057] Because the obstacle monitoring system is configured as described above, by sequentially uploading field-related information, work-related information, and location information to the cloud server 700 via the external mobile terminals 140 present in the fields A, B, and C, it becomes possible to create farm work plans and work route information using cloud computing on the control device 135 and personal computer 610 mounted on the work vehicle 10. In this way, the obstacle monitoring system according to this embodiment is expected to provide agricultural work support from a more multifaceted perspective.

[0058] Furthermore, in this embodiment, using such necessary information, it becomes possible to easily generate work route information that indicates a work route 300 for automatically driving the work vehicle 10, as shown in Fig. 1. The work route information generated in this manner is detailed and suited to the work content and the conditions of the fields A, B, and C, etc.

[0059] In addition, the cloud server 700 shares information with the farm management servers 900 of other farming organizations by mutually communicating farm management information data in shp format or XML using FMIS via a communication network.

[0060] Next, the obstacle monitoring system will be described in detail with reference to the control flow diagrams of Figures 9 and 10 and Figure 6. Note that the farm field A will be described as a representative example.

[0061] First, the flow of the entire obstacle monitoring system will be described based on the control flow diagram of FIG.

[0062] If the in-field fixed sensor 500 does not detect the intrusion of the obstacle 800 into the field A, monitoring continues.

[0063] When the in-field fixed sensor 500 detects the intrusion of an obstacle 800 into the field A, it notifies the cloud server 700 of the intrusion of the obstacle 800, that is, a moving obstacle detection abnormality (transmits information that the obstacle 800 has intruded).

[0064] Then, if the obstacle sensor 130 of the work vehicle 10 detects an obstacle 800, the work vehicle 10 makes an emergency stop, and notifies the cloud server 700 that the work vehicle 10 has made an emergency stop and that work has been discontinued.

[0065] If the obstacle sensor 130 of the work vehicle 10 does not detect an obstacle 800, the control device 135 reduces the speed of the work vehicle 10 and continues work.

[0066] Then, the cloud server 700 is notified that the work vehicle speed has been changed to low speed driving and that the work process has been changed (change in work time in field A), and the work continues.

[0067] When all work processes in field A have been completed, the obstacle monitoring system in field A is terminated.

[0068] If all work processes in field A have not been completed and the in-field fixed sensor 500 has not detected the exit (escape) of the obstacle 800 that has entered field A from field A, the process returns to the routine to determine whether the obstacle sensor 130 of the work vehicle 10 has detected the obstacle 800.

[0069] If all work processes in field A have not been completed and the in-field fixed sensor 500 detects that an obstacle 800 that has entered field A has left (escaped from) field A, it is checked whether the obstacle sensor 130 of the work vehicle 10 has detected the obstacle 800.

[0070] Then, if the obstacle sensor 130 of the work vehicle 10 does not detect the obstacle 800, it is checked whether the work vehicle 10 is stopped, and if the work vehicle 10 is stopped, the work vehicle 10 is restarted to resume work, and the cloud server 700 is notified of the resumption of operation of the work vehicle 10 and of the change in the work process (change in work hours in field A).

[0071] If the work vehicle 10 is not stopped, the speed of the work vehicle 10 is returned from a low speed to a standard speed, and the cloud server 700 is notified of the change in the standard speed of the work vehicle 10 and the change in the work process (change in the work time in field A).

[0072] Then, when all work processes in field A have been completed, the obstacle monitoring system in field A is terminated, the work vehicle 10 is moved to the next field D to work in field D, and the system flow returns to the start point and the obstacle monitoring system in field D is started.

[0073] Furthermore, if the obstacle sensor 130 of the work vehicle 10 detects an obstacle 800, work continues; if all work processes in field A have not been completed, the process returns to the routine for checking whether the in-field fixed sensor 500 has detected the exit (escape) of the obstacle 800 that has entered field A from field A; if all work processes in field A have been completed, the obstacle monitoring system in field A is terminated.

[0074] Next, a case where an obstacle 800 collides with or comes into contact with the work vehicle 10 will be described with reference to the control flow diagram of FIG.

[0075] An obstacle 800 colliding with a work vehicle 10 means, for example, that the obstacle 800 is an animal, a person, or another work vehicle 10 that is approaching the work vehicle 10 and collides with it or slides under the vehicle.

[0076] An example of a situation in which the obstacle 800 comes into contact with the work vehicle 10 is when the obstacle 800 is a person approaching the work vehicle 10 and operating the work vehicle 10 with the intent of theft.

[0077] If the in-field fixed sensor 500 does not detect the intrusion of the obstacle 800 into the field A, monitoring continues.

[0078] When the in-field fixed sensor 500 detects the intrusion of an obstacle 800 into the field A, it notifies the cloud server 700 of the intrusion of the obstacle 800, that is, a moving obstacle detection abnormality (transmits information that the obstacle 800 has intruded).

[0079] Then, if the obstacle sensor 130 of the work vehicle 10 detects an obstacle 800, the work vehicle 10 makes an emergency stop, and notifies the cloud server 700 that the work vehicle 10 has made an emergency stop and that work has been discontinued.

[0080] If the obstacle sensor 130 of the work vehicle 10 does not detect an obstacle 800, the control device 135 reduces the speed of the work vehicle 10 and continues work.

[0081] Then, if the obstacle sensor 130 of the work vehicle 10 detects an obstacle 800 and it suddenly disappears within the detection range of the obstacle sensor 130, if the obstacle sensor 130 of the work vehicle 10 detects an obstacle 800 and the size of the obstacle 800 is larger than a predetermined value, or if the obstacle sensor 130 of the work vehicle 10 detects an obstacle 800 and it is approaching the work vehicle 10, the work vehicle 10 is brought to an emergency stop and moving obstacle abnormality information is sent to the cloud server 700.

[0082] The cloud server 700 receives the moving obstacle abnormality information and communicates the moving obstacle abnormality information to each terminal.

[0083] The worker (supervisor) confirms the moving obstacle abnormality information, goes to the corresponding work vehicle 10 in field A, and takes action.

[0084] When the in-field fixed sensor 500 detects the intrusion of an obstacle 800 into the fields A, B, and C, the detection level (detection time, detection range) of the obstacle sensor 130 throughout the fields A, B, and C is increased.

[0085] Then, when the fixed sensor confirms that the obstacle 800 has retreated from the fields A, B, and C, the detection degree (detection time, detection range) of the obstacle sensor 130 in the entire fields A, B, and C is returned to standard.

[0086] In addition, the cloud server 700 predicts the destination of the obstacle 800 based on the movement information of the obstacle 800.

[0087] Furthermore, the cloud server 700 averages the degree of work progress for each field by preferentially allocating the work vehicle 10 to fields A, B, and C where work is delayed.

[0088] In addition, if any combination of two or more of the following occurs: an obstacle sensor 130 detects an obstacle, an imaging means mounted on the work vehicle 10 detects an obstacle, a tilt of the work vehicle 10 of a certain level or more is detected based on data from an ultrasonic sonar mounted on the work vehicle 10, or a tilt of the work vehicle 10 of a certain level or more is detected based on data from a gyro sensor mounted on the work vehicle 10 that determines the attitude of the work vehicle 10, then the control device 135 mounted on the work vehicle 10 will reduce the vehicle speed or stop the vehicle.

[0089] In addition, the cloud server 700 combines and processes complex data into a common platform (software). Since it can use multiple data formats with different extensions, it can create processed data that is compatible with multiple software programs.

[0090] Then, by linking data via FMIS and using cloud data, it can be used for seedling cultivation and as map data.

[0091] In addition, the control device 135 mounted on the work vehicle 10 acquires raw data from the equipment and inputs and outputs this as USB data. The work vehicle 10 is subject to vibrations, and communication may be in the shadow of the driver, etc., so the USB data method is effective.

[0092] Furthermore, the work vehicle 10 is configured to be capable of exchanging data using a CSV file in a unified format regardless of the model of the work vehicle, such as a tractor, combine harvester, rice transplanter, or riding cultivation machine.

[0093] The field data includes the field name, the field area, the field representative point, the work mode, the work implement information, the AB information for automatic driving, and the field area information.

[0094] In addition, the farm management system information, the information from the external mobile terminal 140 in the field, and the information from the control device 135 mounted on the work vehicle 10 are collated and shared and recorded as field information using the FMIS.

[0095] In addition, to confirm a field, if two or more of the following items are the same: field name, field area, or field representative point, it is recognized as the same field and is shared and recorded.

[0096] In summary, the obstacle monitoring system comprises an obstacle sensor 130 mounted on the work vehicle 10, a control device 135 mounted on the work vehicle 10, an external mobile terminal 140 such as a tablet terminal as an information processing device that can be carried onto the work vehicle 10, a fixed sensor 500 in the field that detects intrusion into and exit from the fields A, B and C, a personal computer 610 as an information processing terminal provided in the ground base 600, and a cloud server 700.

[0097] The in-field fixed sensor 500 monitors the intrusion and exit (entry and exit) of an obstacle 800 into the fields A, B, and C, and transmits intrusion information to the cloud server 700 upon confirming the intrusion of an obstacle 800.

[0098] The obstacle sensor 130 of the work vehicle 10 transmits information about the presence or absence of an obstacle 800 from the control device 135 mounted on the work vehicle 10 to the external mobile terminal 140 and the cloud server 700 .

[0099] If the fixed sensor 500 in the field confirms the intrusion of an obstacle 800 into fields A, B, and C, and the obstacle sensor 130 of the work vehicle 10 detects the obstacle 800, the control device 135 installed in the work vehicle 10 stops the work vehicle 10 and transmits a notification of the work process to the cloud server 700, and the cloud server 700 transmits a notification of the work process to each terminal 140, 610.

[0100] If the in-field fixed sensor 500 detects the intrusion of an obstacle 800 into the fields A, B, and C, but the obstacle sensor 130 of the work vehicle 10 does not detect the obstacle 800, the vehicle speed of the work vehicle 10 is reduced, the work process time is changed based on the reduction in vehicle speed, and a notification of the change in the work process is sent to the cloud server 700, and the cloud server 700 transmits a notification of the change in the work process to each terminal 140, 610. Therefore, information on obstacles 800 present in fields A, B, and C can be integrated into the cloud server 700 and used to revise ongoing work plans.

[0101] If an obstacle 800 enters fields A, B, or C, it is a condition requiring attention and is immediately reported to the worker (supervisor), but if the obstacle sensor 130 of the work vehicle 10 does not detect it, there is no obstacle 800 in the vicinity of the work vehicle 10, so the work speed is reduced to avoid significant delays in the work. By correctly and instantly analyzing the information on the obstacle 800 in this way, accidents can be prevented while also correcting the work plan, making driving of the autonomous work vehicle 10 close to the level of manned driving.

[0102] Furthermore, after the in-field fixed sensor 500 confirms the intrusion of an obstacle 800 into fields A, B, C, it confirms that the obstacle 800 has left fields A, B, C, and if the obstacle sensor 130 of the work vehicle 10 changes from a state of detecting the obstacle 800 to a state of not detecting it, the operation of the stopped work vehicle 10 is started, the work speed is set to the set speed, the work process time is changed based on the time during which work was down, and a notification of the change in work process is sent to the cloud server 700, and the cloud server 700 sends a notification of the change in work process to each terminal 140, 610.

[0103] After the in-field fixed sensor 500 confirms the intrusion of an obstacle 800 into fields A, B, C, it confirms that the obstacle 800 has left fields A, B, C, and if the obstacle sensor 130 of the work vehicle 10 does not detect the obstacle 800, the reduced work speed is returned to the set work speed, the work process time is changed, and a notification of the change in work process is sent to the cloud server 700, and the cloud server 700 sends a notification of the change in work process to each terminal 140, 610.

[0104] Therefore, even if the obstacle 800 is an animal, it is unlikely that the animal will stay in the field for a long time, and once it is confirmed that the animal has left fields A, B, and C, work can be resumed, or if the work speed has been reduced, the work speed can be returned to normal, allowing for the same response as with manned monitoring.

[0105] In addition, if the obstacle sensor 130 of the work vehicle 10 detects an obstacle 800 and suddenly loses information within the detection range of the obstacle sensor 130, if the size of the obstacle 800 exceeds a specified dimension, or if the obstacle 800 is approaching the work vehicle 10 at a relative speed, the work vehicle 10 stops and transmits moving obstacle abnormality information to the cloud server 700, and the cloud server 700 transmits the moving obstacle abnormality information to each terminal 140, 610.

[0106] Therefore, if the obstacle 800 is determined to be a large animal or a human, a corresponding action can be taken. For example, any of the following cases may result in an accident: an accident involving an obstacle, an accident resulting in injury to a person, a theft from a person, an accident resulting in property damage, a prank, or a hit-and-run. When the vehicle stops, information at that time is sent, and if it becomes a serious accident, it can be secured as final data.

[0107] In addition, the FMIS is a system that adds work data and farm management information data accumulated from past work from the cloud server 700, processes the data using a personal computer 610, and centrally manages the data detected by each sensor of the work vehicle 10. The cloud server 700 is capable of communicating with each terminal 140, 610.

[0108] The cloud server 700 then immediately collates the FMIS data, manages the position information of the work vehicle 10, the obstacle 800, and each of the fields A, B, and C, performs mapping processing, and transmits the mapping processing information to each of the terminals 140,610.

[0109] Therefore, by managing the position information of the work vehicle 10 and obstacles 800 using a mapping diagram, this can be expanded to manage information on work in other fields A, B, and C.

[0110] There is also a possibility that this could lead to crimes or accidents, so by instantly obtaining sensor data such as fuel, oil temperature, battery level, speed, gear range, wheel rotation speed of the work vehicle 10, and obstacle detection from the work vehicle 10, or by comparing this with accumulated data from the external mobile terminal 140, the data can be used as analysis data in the event of an accident, etc.

[0111] In addition, by enabling communication via FMIS, a system that centrally manages existing processed data and real-time detection data, obstacles 800 can be instantly integrated into existing mapping data, making it possible to remotely respond to changes in work processes (work times) in real time even with an autonomous work vehicle 10.

[0112] In addition, when there are workers in fields A, B, and C, they can view this integrated mapping data on the external mobile terminal 140, so that they can check with the cloud server 700 for any changes to the work process, and instantly check the changes to the work process on-site. [Explanation of symbols]

[0113] 10 Work vehicles 130 Obstacle Sensor 135 Control device 140 External mobile terminal 500 Workplace sensor (fixed sensor in the field) 610 Personal Computers 700 Cloud Servers A, B, C Work areas (fields)

Claims

1. An obstacle monitoring system including an obstacle sensor (130) and a control device (135) mounted on a work vehicle (10), an external mobile terminal (140), a work site sensor (500) for detecting intrusion into and exit from a work site (A, B, C), and a cloud server (700), The work site sensor (500) monitors the intrusion and exit of an obstacle (800) into the work site (A, B, C), and when the intrusion of the obstacle (800) is confirmed, transmits intrusion information to the cloud server (700); The obstacle sensor (130) of the work vehicle (10) transmits information on the presence or absence of an obstacle (800) from a control device (135) mounted on the work vehicle (10) to an external mobile terminal (140) and a cloud server (700), When the work site sensor (500) confirms the intrusion of an obstacle (800) into the work site (A, B, C) and the obstacle sensor (130) of the work vehicle (10) detects the obstacle (800), the control device (135) mounted on the work vehicle (10) stops the work vehicle (10) and transmits a work process suspension to the cloud server (700), and the cloud server (700) transmits a work process suspension to the terminal (140); When a work site sensor (500) confirms the intrusion of an obstacle (800) into a work site (A, B, C) but an obstacle sensor (130) of a work vehicle (10) does not detect the obstacle (800), the obstacle monitoring system reduces the speed of the work vehicle (10), changes the work process time based on the reduction in vehicle speed, sends a notification of the change in work process to a cloud server (700), and the cloud server (700) sends a notification of the change in work process to a terminal (140).

2. After the work site sensor (500) confirms that an obstacle (800) has entered the work site (A, B, C), it confirms that the obstacle (800) has left the work site (A, B, C), and if the obstacle sensor (130) of the work vehicle (10) goes from detecting the obstacle (800) to not detecting it, it starts operation of the stopped work vehicle (10), sets the work speed to a set speed, changes the work process time based on the time during which work was stopped, sends a notification of the change in the work process to the cloud server (700), and the cloud server (700) sends a notification of the change in the work process to the terminal (140); The obstacle monitoring system of claim 1, characterized in that after the work site sensor (500) confirms the intrusion of an obstacle (800) into the work site (A, B, C), it confirms that the obstacle (800) has left the work site (A, B, C), and if the obstacle sensor (130) of the work vehicle (10) does not detect the obstacle (800), the reduced work speed is returned to the set work speed, the work process time is changed, and a notification of the change in the work process is sent to the cloud server (700), and the cloud server (700) sends a notification of the change in the work process to the terminal (140).

3. The obstacle monitoring system of claim 1, characterized in that if the obstacle sensor (130) of the work vehicle (10) detects an obstacle (800) and then suddenly loses information within the detection range of the obstacle sensor (130), if the size of the obstacle (800) exceeds a predetermined dimension, or if the obstacle (800) is approaching the work vehicle (10) at a relative speed, the work vehicle (10) is stopped and moving obstacle abnormality information is transmitted to the cloud server (700), and the cloud server (700) transmits the moving obstacle abnormality information to the terminal (140).

4. 4. An obstacle monitoring system as claimed in any one of claims 1 to 3, characterized in that the FMIS is a system that centrally manages data processed by a personal computer (610) by adding work data and farm management information data accumulated from past work from a cloud server (700) and data detected by each sensor of the work vehicle (10), and the cloud server (700) communicates with each terminal (135, 140, 610), the cloud server (700) instantly collates FMIS data, manages position information of the work vehicle (10), obstacles (800), and each work site (A, B, C), performs mapping processing, and transmits the mapping processing information to each terminal (135, 140, 610).

Images