Work system

Abstract

To provide a good work system where work is carried out using a remotely controlled work machine.SOLUTION: A work system 10, in which a remotely operated work machine 12 is used to perform work to eliminate a work target Tg in a work area Rg, includes a management device 16 having an image generation unit 108 that generates a virtual space image Vs corresponding to the real space around a work vehicle 14 equipped with the work machine 12, and a machine remote operation device 18 having a display control unit 142 that displays the virtual space image Vs generated by the image generation unit 108 on a display unit 120. The image generation unit 108 generates a virtual space image Vs including a virtual image Vi corresponding to the work machine 12, and emphasizes the work target Tg.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a work system. [Background technology]

[0002] Patent Document 1 discloses a virtual space display system. In this system, a passenger in an automatically driven vehicle wears a head-mounted display. The head-mounted display displays a virtual space image that differs from the actual environment around the vehicle to the passenger in the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6232649 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Patent Document 1 merely discloses the display of a virtual space that is different from the real environment. Recently, there has been a demand for providing a good working system.

[0005] The present invention aims to solve the above-mentioned problems. [Means for solving the problem]

[0006] An aspect of the present disclosure is a work system in which work can be performed using remotely operated work equipment, comprising: an image generation unit that generates a virtual space image corresponding to the real space around a work vehicle equipped with the work equipment; and a display control unit that displays the virtual space image generated by the image generation unit on a display unit, wherein the image generation unit generates the virtual space image including a virtual image corresponding to the work equipment. [Effects of the Invention]

[0007] According to the present invention, a good working system can be provided. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a work system and work performed by work equipment. [Figure 2] FIG. 2 is a functional block diagram of the working vehicle. [Figure 3] FIG. 3 is a functional block diagram of the management device. [Figure 4] FIG. 4 is a functional block diagram of the equipment remote control device. [Figure 5] FIG. 5 is a functional block diagram of the mobile object remote control device. [Figure 6] 6A and 6B are diagrams illustrating examples of virtual space images. [Figure 7] FIG. 7 is a diagram illustrating an example of a virtual space image. [Figure 8] 8A and 8B are flowcharts showing an example of the operation of the working vehicle. [Figure 9] 9A and 9B are flowcharts showing an example of the operation of the management device. [Figure 10] 10A and 10B are flowcharts showing an example of the operation of the equipment remote control device. [Figure 11] 11A and 11B are flowcharts showing an example of the operation of the mobile object remote control device. [Figure 12] FIG. 12 is a functional block diagram of the management device. [Figure 13] FIG. 13 is a diagram illustrating an example of equipment operator applicant data. [Figure 14] FIG. 14 is a diagram illustrating an example of the moving object operator desirer data. [Figure 15] FIG. 15 is a flowchart illustrating an example of the operation of the management device. [Figure 16] FIG. 16 is a functional block diagram of the management device. DETAILED DESCRIPTION OF THE INVENTION

[0009] FIG. 1 is a diagram illustrating a work system 10 and work performed by work equipment 12. In the work system 10, a predetermined work can be performed by remotely controlled work equipment 12. The predetermined work is the extermination of a work target Tg in a work area Rg. The work target Tg that is the target of the predetermined work is, for example, pests, vermin, etc. The work area Rg is, for example, farmland, forest, garden, etc. In the example shown in FIG. 1, the work equipment 12 performs the work of exterminating the work target Tg, which is a pest that has infested crops in the work area Rg, which is farmland.

[0010] The work equipment 12 shown in Fig. 1 is an air gun. The air gun is aimed at a work target Tg, and a bullet is fired to hit the work target Tg, thereby exterminating the work target Tg. In this embodiment, the work equipment 12 is an air gun, but is not limited to this. The work equipment 12 may also be, for example, a fly swatter, a hunting gun, etc.

[0011] The work system 10 includes a mobile work vehicle 14, a management device 16, an equipment remote control device 18, and a mobile vehicle remote control device 20. The mobile work vehicle 14, the management device 16, the equipment remote control device 18, and the mobile vehicle remote control device 20 are connected to a communication line 22 such as the Internet. The mobile work vehicle 14 shown in FIG. 1 is a vehicle. In this embodiment, the mobile work vehicle 14 is a vehicle, but is not limited to this. The mobile work vehicle 14 may be, for example, a drone, a walking robot, or the like. The mobile work vehicle 14 is equipped with the above-mentioned work equipment 12 and an imaging device 24. The work equipment 12 is equipped with an imaging device 26.

[0012] The imaging device 24 and the imaging device 26 may both be configured with cameras. Both the imaging device 24 and the imaging device 26 acquire real space images, which are images of the real space around the work vehicle 14. The imaging device 24 acquires real space images in the direction in which the work vehicle 14 is moving. The real space images acquired by the imaging device 24 are used for remotely controlling the work vehicle 14, as will be described later.

[0013] The imaging device 26 captures a real space image in the direction in which the work equipment 12 is facing. A virtual space image corresponding to the real space image captured by the imaging device 26 is used for remotely controlling the work equipment 12, as will be described later. The virtual space image is generated by the management device 16.

[0014] The equipment remote control device 18 is operated by the equipment operator Pe. The equipment operator Pe remotely controls the work equipment 12 by operating the equipment remote control device 18. The equipment operator Pe causes the work equipment 12 to perform work to exterminate the work target Tg while viewing the virtual space image acquired from the management device 16. Even in cases where it is difficult for the crew of the work vehicle 14 to operate the work equipment 12, the equipment operator Pe can easily operate the work equipment 12 by remote control from anywhere in the world.

[0015] The mobile body remote control device 20 is remotely operated by a mobile body operator Pm. The mobile body operator Pm remotely controls the work vehicle 14 by operating the mobile body remote control device 20. The mobile body operator Pm moves the work vehicle 14 while viewing real space images acquired from the work vehicle 14 via the management device 16. Even in cases where it is difficult for a crew member to operate the work vehicle 14, the mobile body operator Pm can easily operate the work vehicle 14 by remote control from anywhere in the world.

[0016] 2 is a functional block diagram of the work vehicle 14. The work vehicle 14 has a detection unit 40, a drive unit 42, a communication unit 44, a calculation unit 46, and a memory unit 48. The detection unit 40 detects behavior related to the movement of the work vehicle 14. The detection unit 40 outputs behavior information based on the detection results of the behavior to the calculation unit 46. The detection unit 40 is configured with, for example, an encoder, an acceleration sensor, a gyro sensor, a positioning sensor, etc.

[0017] The drive unit 42 may be configured, for example, with a battery, an electric motor, an internal combustion engine, a power transmission mechanism, left and right wheels, etc. The power transmission mechanism transmits power from the electric motor or the internal combustion engine to the left and right wheels. The communication unit 44 may be configured, for example, with a wireless communication module equipped with an antenna, etc. The communication unit 44 may transmit signals to the outside of the work vehicle 14. The communication unit 44 may also receive signals from the outside of the work vehicle 14.

[0018] The calculation unit 46 may be configured by a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). That is, the calculation unit 46 may be configured by processing circuitry. At least a part of the calculation unit 46 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array). At least a part of the calculation unit 46 may be realized by an electronic circuit including discrete devices.

[0019] The storage unit 48 is a computer-readable storage medium. The storage unit 48 is configured with a volatile memory (not shown) and a non-volatile memory (not shown). The volatile memory is, for example, a RAM (Random Access Memory). The volatile memory is used as a working memory for the processor.

[0020] The nonvolatile memory is, for example, a ROM (Read Only Memory), a flash memory, or the like. The nonvolatile memory stores the programs executed by the processor and other necessary data. In other words, the working vehicle 14 has a program product of the programs executed by the processor. At least a portion of the storage unit 48 may be provided in the above-mentioned processor, integrated circuit, or the like.

[0021] The calculation unit 46 has a mobile object control unit 60, an equipment control unit 62, and an image acquisition unit 64. The calculation unit 46 executes a program stored in the storage unit 48, thereby realizing the mobile object control unit 60, the equipment control unit 62, and the image acquisition unit 64.

[0022] The mobile object control unit 60 acquires the behavior information output from the detection unit 40. Based on the acquired behavior information, the mobile object control unit 60 generates movement information related to the movement status of the work mobile object 14. The movement information includes information indicating the current position and movement speed of the work mobile object 14. The mobile object control unit 60 transmits the generated movement information to the management device 16 via the communication unit 44 and the communication line 22.

[0023] The mobile object control unit 60 acquires a mobile object operation signal corresponding to the operation of the work mobile object 14 by the mobile object operator Pm from the management device 16. The mobile object control unit 60 controls operations related to the movement of the work mobile object 14 in accordance with the acquired mobile object operation signal. The equipment control unit 62 acquires an equipment operation signal corresponding to the operation of the work equipment 12 by the equipment operator Pe from the management device 16. In accordance with the acquired equipment operation signal, the equipment control unit 62 controls the work equipment 12 to cause the work equipment 12 to perform a predetermined task.

[0024] The image acquisition unit 64 acquires a real space image from the imaging device 24. The image acquisition unit 64 acquires a real space image from the imaging device 26. The image acquisition unit 64 transmits image signals of the real space images acquired from the imaging device 24 and the imaging device 26 to the management device 16 via the communication unit 44 and the communication line 22.

[0025] 3 is a functional block diagram of the management device 16. The communication unit 80 may be configured, for example, by a communication module having a connection interface for a communication cable. The communication unit 80 may transmit signals to the outside of the management device 16. The communication unit 80 may also receive signals from the outside of the management device 16.

[0026] The calculation unit 82 may be configured by a processor such as a CPU or a GPU. That is, the calculation unit 82 may be configured by a processing circuit. At least a part of the calculation unit 82 may be realized by an integrated circuit such as an ASIC or an FPGA. At least a part of the calculation unit 82 may be realized by an electronic circuit including discrete devices.

[0027] The memory unit 84 is a computer-readable storage medium. The memory unit 84 is composed of a volatile memory (not shown) and a non-volatile memory (not shown). The volatile memory is, for example, a RAM. The volatile memory is used as a working memory for the processor. The non-volatile memory is, for example, a ROM, a flash memory, etc. The non-volatile memory stores programs executed by the processor, image data, etc. In other words, the management device 16 has a program product of the program executed by the processor. At least a portion of the memory unit 84 may be provided in the above-mentioned processor, integrated circuit, etc.

[0028] The calculation unit 82 has a mobile object operation signal acquisition unit 100, an equipment operation signal acquisition unit 102, a movement information relay unit 104, an image recognition unit 106, and an image generation unit 108. When the calculation unit 82 executes a program stored in the memory unit 84, the mobile object operation signal acquisition unit 100, the equipment operation signal acquisition unit 102, the image recognition unit 106, and the image generation unit 108 can be realized.

[0029] The mobile object operation signal acquisition unit 100 acquires a mobile object operation signal from the mobile object remote control device 20 via the communication line 22 and the communication unit 80. The mobile object operation signal acquisition unit 100 transmits the acquired mobile object operation signal to the work mobile object 14 via the communication unit 80 and the communication line 22. The equipment operation signal acquisition unit 102 acquires an equipment operation signal from the equipment remote control device 18 via the communication line 22 and the communication unit 80. The equipment operation signal acquisition unit 102 transmits the acquired equipment operation signal to the work mobile object 14 via the communication unit 80 and the communication line 22.

[0030] The movement information relay unit 104 acquires movement information generated by the work vehicle 14 from the work vehicle 14 via the communication line 22 and the communication unit 80. The movement information relay unit 104 transmits the acquired movement information to the mobile object remote control device 20 via the communication unit 80 and the communication line 22.

[0031] The image recognition unit 106 acquires an image signal of the real space image acquired by the imaging device 24 and an image signal of the real space image acquired by the imaging device 26 from the working vehicle 14 via the communication line 22 and the communication unit 80. The image recognition unit 106 transmits the image signal of the real space image acquired by the imaging device 24 to the mobile object remote control device 20 via the communication unit 80 and the communication line 22. The image recognition unit 106 recognizes a part corresponding to the work target Tg in the real space image acquired by the imaging device 26.

[0032] The image generation unit 108 performs image processing on the part recognized by the image recognition unit 106. This image processing is, for example, image processing that emphasizes the work object Tg. This image processing may also be image processing that superimposes an image corresponding to the work object Tg on the work object Tg. Specific examples will be described later with reference to FIGS. 6A, 6B, and 7. This allows the equipment operator Pe to easily recognize the work object Tg even if the work object Tg is difficult to identify with the naked eye.

[0033] The image generation unit 108 generates a virtual space image corresponding to the real space image acquired by the imaging device 26. The virtual space image is an image including a virtual image corresponding to the work equipment 12. The virtual image is determined using image data stored in the storage unit 84. If the work equipment 12 is an air gun, for example, an image resembling a rifle target scope is used as the virtual image. If the work equipment 12 is a fly swatter, for example, an image resembling a large hand is used as the virtual image. This allows the equipment operator Pe to enjoy even mundane work, like playing a game. Therefore, an excellent work system 10 can be provided.

[0034] The image generation unit 108 transmits an image signal of the generated virtual space image to the equipment remote control device 18 via the communication unit 80 and the communication line 22.

[0035] 4 is a functional block diagram of the equipment remote control device 18. The equipment remote control device 18 has a display unit 120, an operation detection unit 122, a communication unit 124, a calculation unit 126, and a storage unit 128.

[0036] The display unit 120 displays a virtual space image according to an image signal transmitted from the management device 16. The display unit 120 is, for example, a display (monitor) of a computer such as a PC (Personal Computer) or a smartphone. The display unit 120 may be a head-up display. The display unit 120 may be a head-mounted display such as AR (Augmented Reality) goggles. However, the display unit 120 is not limited to these displays.

[0037] The operation detection unit 122 may be configured, for example, by an operator (lever, button, etc.) that can be operated by the equipment operator Pe, and a sensor that detects and outputs the operation of the operator (amount of operation, direction of operation, etc.). The operation detection unit 122 outputs an equipment operation signal corresponding to the operation of the operator to the calculation unit 126.

[0038] The communication unit 124 may be configured, for example, by a communication module having a connection interface for a communication cable. The communication unit 124 may transmit signals to the outside of the equipment remote control device 18. The communication unit 124 may also receive signals from the outside of the equipment remote control device 18.

[0039] The calculation unit 126 may be configured by a processor such as a CPU or a GPU. That is, the calculation unit 126 may be configured by a processing circuit. At least a part of the calculation unit 126 may be realized by an integrated circuit such as an ASIC or an FPGA. At least a part of the calculation unit 126 may be realized by an electronic circuit including discrete devices.

[0040] The memory unit 128 is a computer-readable storage medium. The memory unit 128 is composed of a volatile memory (not shown) and a non-volatile memory (not shown). The volatile memory is, for example, a RAM. The volatile memory is used as a working memory for the processor. The non-volatile memory is, for example, a ROM or a flash memory. The non-volatile memory stores programs executed by the processor and other necessary data. In other words, the equipment remote control device 18 has a program product of the programs executed by the processor. At least a portion of the memory unit 128 may be provided in the above-mentioned processor, integrated circuit, etc.

[0041] The calculation unit 126 has an image signal acquisition unit 140, a display control unit 142, and an equipment remote control unit 144. The calculation unit 126 executes a program stored in the storage unit 128, thereby realizing the image signal acquisition unit 140, the display control unit 142, and the equipment remote control unit 144.

[0042] The image signal acquisition unit 140 acquires an image signal of a virtual space image generated by the management device 16 from the management device 16 via the communication line 22 and the communication unit 124. The display control unit 142 causes the display unit 120 to display a virtual space image corresponding to the image signal acquired by the image signal acquisition unit 140. The equipment remote operation unit 144 acquires an equipment operation signal output by the operation detection unit 122 and transmits it to the management device 16 via the communication unit 124 and the communication line 22.

[0043] 5 is a functional block diagram of the mobile object remote control device 20. The mobile object remote control device 20 has a display unit 160, an operation detection unit 162, a communication unit 164, a calculation unit 166, and a storage unit 168. The display unit 160 displays a real space image corresponding to an image signal transmitted from the management device 16. The real space image is an image acquired by the imaging device 24, and corresponds to the real space around the work vehicle 14.

[0044] The display unit 160 is, for example, a display (monitor) of a computer such as a PC (Personal Computer) or a smartphone. The display unit 160 may be a head-up display. The display unit 160 may be a head-mounted display such as AR (Augmented Reality) goggles. However, the display unit 160 is not limited to these displays.

[0045] The operation detection unit 162 may be configured, for example, by an operator (lever, button, etc.) that can be operated by the mobile object operator Pm, and a sensor that detects and outputs the operation of the operator (operation amount, operation direction, etc.). The operation detection unit 162 outputs a mobile object operation signal corresponding to the operation of the operator to the calculation unit 166.

[0046] The communication unit 164 may be configured, for example, by a communication module having a connection interface for a communication cable. The communication unit 164 may transmit signals to the outside of the mobile object remote control device 20. The communication unit 164 may also receive signals from the outside of the mobile object remote control device 20.

[0047] The calculation unit 166 may be configured by a processor such as a CPU or a GPU. That is, the calculation unit 126 may be configured by a processing circuit. At least a part of the calculation unit 166 may be realized by an integrated circuit such as an ASIC or an FPGA. At least a part of the calculation unit 166 may be realized by an electronic circuit including discrete devices.

[0048] The storage unit 168 is a computer-readable storage medium. The storage unit 168 is composed of a volatile memory (not shown) and a non-volatile memory (not shown). The volatile memory is, for example, a RAM. The volatile memory is used as a working memory for the processor. The non-volatile memory is, for example, a ROM, a flash memory, etc. The non-volatile memory stores programs executed by the processor and other necessary data. In other words, the mobile object remote control device 20 has a program product of the program executed by the processor. At least a portion of the storage unit 168 may be provided in the above-mentioned processor, integrated circuit, etc.

[0049] The calculation unit 166 has a movement information acquisition unit 180, an image signal acquisition unit 182, a display control unit 184, and a mobile object remote control unit 186. The movement information acquisition unit 180, the image signal acquisition unit 182, the display control unit 184, and the mobile object remote control unit 186 can be realized by the calculation unit 166 executing a program stored in the storage unit 168.

[0050] The movement information acquisition unit 180 acquires the movement information relayed by the management device 16 from the management device 16 via the communication line 22 and the communication unit 164. The display control unit 184 may cause the display unit 160 to display information indicating the current position and movement speed of the work vehicle 14, which is included in the movement information acquired by the movement information acquisition unit 180. In this case, the mobile body operator Pm can operate the control while taking into consideration the information on the current position and movement speed displayed on the display unit 160.

[0051] The image signal acquisition unit 182 acquires the real space image acquired by the imaging device 24 from the management device 16 via the communication line 22 and the communication unit 164. The display control unit 184 causes the display unit 160 to display the real space image corresponding to the image signal acquired by the image signal acquisition unit 182. The mobile object remote control unit 186 acquires the mobile object operation signal output by the operation detection unit 162 and transmits it to the management device 16 via the communication unit 164 and the communication line 22.

[0052] 6A and 6B are diagrams illustrating a virtual space image Vs. In the example shown in FIGS. 6A and 6B, the virtual space image Vs is displayed on the display unit 120 of the equipment remote control device 18. The virtual space image Vs includes a virtual image Vi that resembles a rifle target scope. The virtual image Vi does not overlap with the image of the work object Tg. In other words, the work equipment 12 is not facing the work object Tg.

[0053] Image processing is performed by the image generation unit 108 of the management device 16 on the portion of the virtual space image Vs that corresponds to the work object Tg. In the example shown in FIG. 6A, the image processing performed on the virtual space image Vs is image processing that emphasizes the work object Tg by using an image Hi, which is a bold frame graphic that surrounds the image of the work object Tg. The image of the work object Tg may also be emphasized by enlarging it. In the example shown in FIG. 6B, the image processing performed on the virtual space image Vs is image processing that superimposes an image Ai of a monster on the work object Tg as an image corresponding to the work object Tg. This type of image processing allows the equipment operator Pe to easily grasp the work object Tg.

[0054] FIG. 7 is a diagram illustrating a virtual space image Vs. In the example shown in FIG. 7, the equipment operator Pe remotely controls the work equipment 12 to change the orientation of the work equipment 12, so that the virtual image Vi overlaps the image of the work object Tg. At this time, the equipment operator Pe can perform work to eliminate the work object Tg by remotely controlling the work equipment 12 to act on the work object Tg. If the work equipment 12 is an air gun, the equipment operator Pe can fire a bullet from the air gun, which hits the work object Tg, thereby eliminating the work object Tg. In this way, the equipment operator Pe can easily perform work.

[0055] 8A and 8B are flowcharts showing an example of the operation of the work vehicle 14. FIG. 8A shows the processing procedure when the work vehicle 14 transmits movement information and image signals to the management device 16. This processing procedure is performed by the calculation unit 46 of the work vehicle 14 executing a program stored in the memory unit 48. This processing procedure is performed repeatedly while the work vehicle 14 is being remotely operated.

[0056] When this processing procedure starts, in step S1, the mobile object control unit 60 acquires behavior information output from the detection unit 40. In step S2, the mobile object control unit 60 generates movement information based on the behavior information acquired in step S1, and transmits the generated movement information to the management device 16. In step S3, the image acquisition unit 64 acquires real space images from each of the imaging devices 24 and 26. In step S4, the image acquisition unit 64 transmits image signals of the real space images acquired in step S3 to the management device 16. When the processing of step S4 is completed, this processing procedure ends.

[0057] 8B shows the processing procedure for controlling the movement of the work vehicle 14 and the work equipment 12. This processing procedure is performed by the calculation unit 46 of the work vehicle 14 executing a program stored in the memory unit 48. This processing procedure is performed repeatedly while the work vehicle 14 is being remotely operated.

[0058] When this processing procedure starts, in step S21, the mobile object control unit 60 determines whether or not a mobile object operation signal has been acquired from the management device 16. If the answer is YES in step S21, this processing procedure proceeds to step S22. If the answer is NO in step S21, this processing procedure proceeds to step S23. In step S22, the mobile object control unit 60 controls operations related to the movement of the work mobile object 14 in accordance with the acquired mobile object operation signal.

[0059] In step S23, the equipment control unit 62 determines whether or not an equipment operation signal has been acquired from the management device 16. If the answer is YES in step S23, the processing procedure proceeds to step S24. If the answer is NO in step S23, the processing procedure ends. In step S24, the equipment control unit 62 controls the work equipment 12 in accordance with the acquired equipment operation signal. When the processing of step S24 is completed, the processing procedure ends.

[0060] 9A and 9B are flowcharts showing an example of the operation of the management device 16. Fig. 9A shows the processing procedure when the management device 16 transmits movement information and image signals to the equipment remote control device 18 and the mobile object remote control device 20. This processing procedure is performed by the calculation unit 82 of the management device 16 executing a program stored in the memory unit 84. This processing procedure is performed repeatedly while the work vehicle 14 is being remotely controlled.

[0061] When this processing procedure starts, in step S41, the movement information relay unit 104 acquires movement information from the work vehicle 14. In step S42, the movement information relay unit 104 transmits the movement information acquired in step S41 to the vehicle remote control device 20. In step S43, the image recognition unit 106 acquires, from the work vehicle 14, an image signal of the real space image acquired by the imaging device 24 and an image signal of the real space image acquired by the imaging device 26.

[0062] In step S44, the image recognition unit 106 determines whether or not a part corresponding to the work object Tg has been recognized in the real space image acquired by the imaging device 26. If the answer is YES in step S44, the processing procedure proceeds to step S45. If the answer is NO in step S44, the processing procedure proceeds to step S46. In step S45, the image generation unit 108 performs image processing on the part recognized by the image recognition unit 106.

[0063] In step S46, the image generation unit 108 determines a virtual image Vi corresponding to the work equipment 12 using the image data stored in the storage unit 84. The image generation unit 108 superimposes the determined virtual image Vi on the image obtained by applying image processing to the real space image in step S45. In this way, the image generation unit 108 generates a virtual space image Vs that corresponds to the real space around the work vehicle 14 equipped with the work equipment 12 and includes the virtual image Vi.

[0064] In step S47, the image generation unit 108 transmits an image signal of the virtual space image Vs generated in step S46 to the equipment remote control device 18. In addition, the image recognition unit 106 transmits an image signal of the real space image acquired by the imaging device 24 in step S43 to the mobile object remote control device 20. When the processing of step S47 is completed, this processing procedure ends.

[0065] 9B shows the processing procedure for the mobile unit operation signal and the equipment operation signal by the management device 16. This processing procedure is performed by the calculation unit 82 of the management device 16 by executing a program stored in the memory unit 84. This processing procedure is performed repeatedly while the work vehicle 14 is being remotely operated.

[0066] When this processing procedure starts, in step S61, the mobile object operation signal acquisition unit 100 determines whether or not a mobile object operation signal has been acquired from the mobile object remote control device 20. If the result in step S61 is YES, this processing procedure proceeds to step S62. If the result in step S61 is NO, this processing procedure proceeds to step S63. In step S62, the mobile object operation signal acquisition unit 100 transmits the acquired mobile object operation signal to the work mobile object 14.

[0067] In step S63, the equipment operation signal acquisition unit 102 determines whether or not an equipment operation signal has been acquired from the equipment remote control device 18. If the result in step S63 is YES, the processing procedure proceeds to step S64. If the result in step S63 is NO, the processing procedure ends. In step S64, the equipment operation signal acquisition unit 102 transmits the acquired equipment operation signal to the work vehicle 14. When the processing of step S64 is completed, the processing procedure ends.

[0068] 10A and 10B are flowcharts showing an example of the operation of the equipment remote control device 18. Fig. 10A shows the processing procedure when the equipment remote control device 18 acquires an image signal from the management device 16. This processing procedure is performed by the calculation unit 126 of the equipment remote control device 18 executing a program stored in the memory unit 128. This processing procedure is performed repeatedly while the working vehicle 14 is being remotely controlled.

[0069] When this processing procedure starts, in step S81, image signal acquisition unit 140 acquires, from management device 16, an image signal of virtual space image Vs generated by management device 16. In step S82, display control unit 142 causes display unit 120 to display virtual space image Vs corresponding to the image signal acquired in step S81. When the processing of step S82 is completed, this processing procedure ends.

[0070] 10B shows the processing procedure when the equipment remote control device 18 transmits an equipment operation signal to remotely control the work equipment 12. This processing procedure is performed by the calculation unit 126 of the equipment remote control device 18 executing a program stored in the memory unit 128. This processing procedure is performed repeatedly while the work vehicle 14 is being remotely controlled.

[0071] When this processing procedure starts, in step S101, the equipment remote control unit 144 acquires an equipment operation signal corresponding to the operation of a control by the equipment operator Pe. In step S102, the equipment remote control unit 144 transmits the equipment operation signal acquired in step S101 to the management device 16. When the processing of step S102 is completed, this processing procedure ends.

[0072] 11A and 11B are flowcharts showing an example of the operation of the mobile object remote control device 20. Fig. 11A shows the processing procedure when the mobile object remote control device 20 acquires movement information and an image signal from the management device 16. This processing procedure is performed by the calculation unit 166 of the mobile object remote control device 20 executing a program stored in the memory unit 168. This processing procedure is performed repeatedly while the work vehicle 14 is being remotely controlled.

[0073] When this processing procedure starts, in step S121, the movement information acquisition unit 180 acquires movement information generated by the work vehicle 14 from the management device 16. In step S122, the image signal acquisition unit 182 acquires an image signal of a real space image acquired by the imaging device 24 from the management device 16. In step S123, the display control unit 184 causes the display unit 160 to display a real space image corresponding to the image signal acquired in step S122. When the processing of step S123 is completed, this processing procedure ends.

[0074] 11B shows the processing procedure when the mobile remote control device 20 transmits a mobile control signal to remotely control the work vehicle 14. This processing procedure is performed by the calculation unit 166 of the mobile remote control device 20 executing a program stored in the memory unit 168. This processing procedure is performed repeatedly while the work vehicle 14 is being remotely controlled.

[0075] When this processing procedure starts, in step S141, the mobile object remote control unit 186 acquires a mobile object operation signal corresponding to the operation of the control by the mobile object operator Pm. In step S142, the mobile object remote control unit 186 transmits the mobile object operation signal acquired in step S141 to the management device 16. When the processing of step S142 is completed, this processing procedure ends.

[0076] The above-described embodiment may be modified as follows: In the following modifications, explanations that overlap with the above-described embodiment will be omitted.

[0077] (Variation 1) In the above-described embodiment, the equipment remote control device 18 is operated by an equipment operator Pe. The mobile object remote control device 20 is remotely operated by a mobile object operator Pm. The equipment operator Pe and the mobile object operator Pm may be selected from among the applicants who wish to operate the work equipment 12 and the work mobile object 14, respectively. In this first modification, the applicant operator data for each of the work equipment 12 and the work mobile object 14 is used.

[0078] Fig. 12 is a functional block diagram of the management device 16. In Fig. 12, the same reference numerals are used for components common to Fig. 3. A description of these components will be omitted. The calculation unit 82 shown in Fig. 12 further includes a work information acquisition unit 200 and an operator determination unit 202. The calculation unit 82 executes a program stored in the memory unit 84, thereby realizing the work information acquisition unit 200 and the operator determination unit 202.

[0079] The storage unit 84 also stores in advance data 210 of aspiring equipment operators and data 212 of aspiring mobile unit operators. The data 210 of aspiring equipment operators is data relating to aspiring equipment operators who wish to operate the work equipment 12. The data 212 of aspiring mobile unit operators is data relating to aspiring mobile unit operators who wish to operate the work mobile unit 14. Examples of the data 210 of aspiring equipment operators and the data 212 of aspiring mobile unit operators will be described later with reference to FIGS. 13 and 14, respectively.

[0080] When a requester of a predetermined work inputs work information, which is information related to the predetermined work, into a work information input device (not shown), the work information acquisition unit 200 acquires the work information from the work information input device via the communication line 22 and the communication unit 80. The work information includes, for example, area information indicating that the work area Rg is farmland, geographic information indicating that the work area Rg is on flat land as a geographical condition of the work area Rg, location information related to the position and size of the work area Rg, schedule information indicating the work time and period when the predetermined work will be performed, and work object information related to the work object Tg.

[0081] The operator determination unit 202 further includes an equipment operator determination unit 220 and a mobile object operator determination unit 222. The equipment operator determination unit 220 acquires the aspirant equipment operator data 210 from the storage unit 84. The equipment operator determination unit 220 determines the equipment operator Pe who will operate the work equipment 12 based on the above-mentioned work information and the aspirant equipment operator data 210. In this way, the operator determination unit 202 can determine an appropriate equipment operator Pe.

[0082] The mobile object operator determination unit 222 acquires the mobile object operator desirer data 212 from the storage unit 84. The mobile object operator determination unit 222 determines the mobile object operator Pm who will operate the work mobile object 14 based on the above-described work information and the mobile object operator desirer data 212. In this way, the operator determination unit 202 can determine an appropriate mobile object operator Pm.

[0083] The equipment operator data 210 includes at least one of information on the work equipment 12 that the equipment operator candidate can operate, the work history of the equipment operator candidate, and the skills of the equipment operator candidate. The equipment operator data 210 may further include at least one of information on the work area Rg that the equipment operator candidate can handle and information on the geographical conditions that the equipment operator candidate can handle.

[0084] The equipment operator applicant data 210 may further include at least one of information on location conditions, information on schedule conditions, and information on work target conditions. The location conditions determine the location and size of the work area Rg that the equipment operator applicant can handle. The schedule conditions determine the work time and work period in which the equipment operator applicant can perform a specified work. The work target conditions determine the work target Tg (vermin, vermin, vermin birds, etc.) that the equipment operator applicant can handle.

[0085] Fig. 13 is a diagram illustrating an example of equipment operator aspirant data 210. Fig. 13 illustrates data for a plurality of equipment operator aspirants E1, E2, ..., E5. The equipment operator aspirant data 210 records information about the work equipment 12 that each equipment operator aspirant can operate, such as an air gun, a fly swatter, a hunting rifle, etc. The equipment operator aspirant data 210 also records the number of years of experience working with the work equipment 12 as the work history of each equipment operator aspirant.

[0086] The equipment operator candidate data 210 records the skills of each equipment operator candidate, such as the qualifications, licenses, and tests that each equipment operator candidate holds. The equipment operator candidate data 210 also records farmland, gardens, forests, etc. as the work areas Rg in which the work that each equipment operator candidate has experienced has been performed. The equipment operator candidate data 210 also records flat land, wetlands, slopes, etc. as the geographical conditions of the work areas Rg in which the work that each equipment operator candidate has experienced has been performed.

[0087] 13, location condition information, schedule condition information, and work target condition information are all omitted from the illustration. In reality, the equipment operator determination unit 220 determines the equipment operator Pe after taking into consideration the schedule conditions and the like.

[0088] The mobile object operator desirer data 212 includes at least one of information regarding the work vehicle 14 that the mobile object operator desirer can operate, the mobile object operation history of the mobile object operator desirer, and the skills of the mobile object operator desirer. The mobile object operator desirer data 212 may further include at least one of information regarding the work area Rg that the mobile object operator desirer can handle and information regarding the geographical conditions that the mobile object operator desirer can handle.

[0089] The mobile object operator desirer data 212 may further include at least one of information on location conditions and information on schedule conditions. The location conditions determine the location and size of the work area Rg that the mobile object operator desirer can handle. The schedule conditions determine the work time and work period that the mobile object operator desirer can handle.

[0090] Fig. 14 is a diagram illustrating an example of the mobile object operator desirer data 212. Fig. 14 illustrates data for multiple mobile object operator desirers M1, M2, ..., M5. The mobile object operator desirer data 212 records information about the work vehicle 14 that each mobile object operator desirer can operate, such as a vehicle, a walking robot, or a drone. The mobile object operator desirer data 212 records the number of years of experience operating a work vehicle 14 as the operation history of each mobile object operator.

[0091] The mobile unit operator candidate data 212 records the skills of each mobile unit operator candidate, such as the qualifications, licenses, and exams that each mobile unit operator candidate has. The mobile unit operator candidate data 212 records farmland, gardens, forests, etc. as the work area Rg in which each mobile unit operator candidate operated the work vehicle 14. The mobile unit operator candidate data 212 records flat land, wetlands, slopes, etc. as the geographical conditions of the work area Rg in which each mobile unit operator candidate operated the work vehicle 14.

[0092] Note that neither the location condition information nor the schedule condition information is shown in Fig. 14. In reality, the mobile object operator determination unit 222 determines the mobile object operator Pm after taking into consideration the schedule condition and the like.

[0093] Even if the difficulty of a predetermined task indicated by the task information is high, an appropriate equipment operator Pe and / or mobile object operator Pm can be determined based on the equipment operator candidate data 210 and / or mobile object operator candidate data 212. For example, an operator candidate with many years of experience in operating equipment or mobile objects, or an operator candidate with experience working or operating in a work area Rg with difficult geographical conditions, can be preferentially determined as an appropriate equipment operator Pe and / or mobile object operator Pm.

[0094] Fig. 15 is a flowchart showing an example of the operation of the management device 16. Fig. 15 shows the processing procedure when the operator determination unit 202 determines an equipment operator Pe and a mobile object operator Pm. This processing procedure is performed by the calculation unit 82 of the management device 16 by executing a program stored in the memory unit 84. This processing procedure is performed when work information is input into the work information input device by a requester of a specific work.

[0095] When this processing procedure starts, in step S161, the equipment operator determination unit 220 and the mobile object operator determination unit 222 acquire task information from the task information input device. In step S162, the equipment operator determination unit 220 acquires the aspirant equipment operator data 210 from the storage unit 84. In step S163, the equipment operator determination unit 220 determines an equipment operator Pe based on the task information acquired in step S161 and the aspirant equipment operator data 210 acquired in step S162.

[0096] In step S164, the mobile object operator determination unit 222 acquires the mobile object operator aspirant data 212 from the storage unit 84. In step S165, the mobile object operator determination unit 222 determines the mobile object operator Pm based on the task information acquired in step S161 and the mobile object operator aspirant data 212 acquired in step S164. When the processing of step S164 is completed, this processing procedure ends.

[0097] (Variation 2) In the above-described first modification, the operator determination unit 202 determines an equipment operator Pe and a moving object operator Pm. The number of determined equipment operators Pe and moving object operators Pm is not limited to one each. In the present second modification, a formation consisting of a plurality of equipment operators Pe and a plurality of moving object operators Pm is determined.

[0098] FIG. 16 is a functional block diagram of the management device 16. In FIG. 16, the same reference numerals are used for components common to FIG. 3 or 12. A description of these components will be omitted. The calculation unit 82 shown in FIG. 16 further includes a formation determination unit 240. The calculation unit 82 executes a program stored in the storage unit 84, thereby realizing the formation determination unit 240.

[0099] The formation determination unit 240 acquires the equipment operator aspirant data 210 and the mobile object operator aspirant data 212 from the memory unit 84. The formation determination unit 240 determines a formation consisting of multiple equipment operators Pe and multiple mobile object operators Pm in accordance with the work information acquired by the work information acquisition unit 200. The formation determination unit 240 may further determine the formation, arrangements, and strategy of the multiple work equipment 12 and multiple work mobile objects 14 used in the formation.

[0100] For example, a formation consisting of multiple vehicles, each equipped with an air gun, and a drone equipped with an insecticide sprayer is determined. A formation is determined in which multiple vehicles move in two columns following a leading drone. A sequence is determined in which insecticide is sprayed from the insecticide sprayer attached to the leading drone, and bullets are fired at the pests from the air gun attached to the following vehicle. A strategy is determined that increases the effectiveness of pest extermination by shooting the pests with the air gun once the insecticide has suppressed their movement.

[0101] By determining the formation, a sense of unity can be fostered among the multiple equipment operators Pe and multiple mobile body operators Pm who are members of the formation, which increases the motivation of the equipment operators Pe and the mobile body operators Pm to perform a given task, thereby providing a better work system 10.

[0102] The following additional notes are provided regarding the above-described embodiment and modifications.

[0103] (Appendix 1) A work system (10) in which work can be performed using remotely operated work equipment (12) includes an image generation unit (108) that generates a virtual space image (Vs) corresponding to the real space around a work vehicle (14) equipped with the work equipment, and a display control unit (142) that displays the virtual space image generated by the image generation unit on a display unit (120), the image generation unit generating the virtual space image including a virtual image (Vi) corresponding to the work equipment. With this configuration, an equipment operator who operates the work equipment can enjoy even mundane tasks like playing a game. Therefore, a good work system can be provided.

[0104] (Appendix 2) In the work system described in Supplementary Note 1, the image generation unit may generate the virtual space image in which image processing has been performed on a part corresponding to a work target (Tg) that is the target of the work. With this configuration, even if the work target is difficult to identify with the naked eye, the equipment operator can easily recognize the work target.

[0105] (Appendix 3) In the work system described in Supplementary Note 2, the image processing may be image processing that emphasizes the work object. With this configuration, the equipment operator can easily grasp the work object.

[0106] (Appendix 4) In the work system described in Supplementary Note 2, the image processing may be image processing that superimposes an image (Ai) corresponding to the work object on the work object. With this configuration, the equipment operator can easily grasp the work object.

[0107] (Appendix 5) In the work system described in Supplementary Note 1, the work vehicle may be remotely controlled. With this configuration, even if it is difficult for a crew member to operate the work vehicle, the vehicle operator who operates the work vehicle can easily operate the work vehicle by remote control.

[0108] (Appendix 6) The work system described in Supplementary Note 1 may further include an operator determination unit (202) that determines an equipment operator (Pe) who will operate the work equipment and a mobile body operator (Pm) who will operate the work mobile body based on work information, which is information about the work, equipment operator candidate data (210) related to an applicant who wishes to operate the work equipment, and mobile body operator candidate data (212) related to an applicant who wishes to operate the work mobile body. With this configuration, even if the difficulty level of a predetermined work indicated by the work information is high, an appropriate equipment operator and mobile body operator can be determined.

[0109] (Appendix 7) The work system described in Supplementary Note 1 further comprises an equipment operator determination unit (220) that determines an equipment operator who will operate the work equipment based on work information, which is information about the work, and equipment operator candidate data about an aspirant who wishes to operate the work equipment, wherein the equipment operator candidate data may include at least one of information about the work equipment that the aspirant can operate, the work history of the aspirant, and the skills of the aspirant. With this configuration, an appropriate equipment operator can be determined even if the difficulty of a specific work indicated by the work information is high.

[0110] (Appendix 8) The work system described in Supplementary Note 1 further includes a mobile object operator determination unit (222) that determines a mobile object operator who will operate the work mobile object based on work information, which is information about the work, and mobile object operator desire data about an aspirant mobile object operator who desires to operate the work mobile object, wherein the aspirant mobile object operator data may include at least one of information about the work mobile object that the aspirant mobile object operator can operate, the mobile object operation history of the aspirant mobile object operator, and the skills of the aspirant mobile object operator. With this configuration, an appropriate mobile object operator can be determined even if the difficulty of the predetermined work indicated by the work information is high.

[0111] (Appendix 9) The work system described in Supplementary Note 1 further includes an equipment operator determination unit that determines an equipment operator who will operate the work equipment based on work information, which is information about the work, and equipment operator candidate data about an aspirant who wishes to operate the work equipment, wherein the equipment operator candidate data may include at least one of information about a work area (Rg) that the aspirant can handle and information about geographical conditions that the aspirant can handle. With this configuration, an appropriate equipment operator can be determined even if the difficulty of a specific work indicated by the work information is high.

[0112] (Appendix 10) The work system described in Supplementary Note 1 further includes a mobile object operator determination unit that determines a mobile object operator who will operate the work mobile object based on work information, which is information about the work, and desired mobile object operator data about an aspirant mobile object operator who wishes to operate the work mobile object, and the desired mobile object operator data may include at least one of information about a work area that the aspirant mobile object operator can handle and information about geographical conditions that the aspirant mobile object operator can handle. With this configuration, an appropriate mobile object operator can be determined even if the difficulty of the predetermined work indicated by the work information is high.

[0113] (Appendix 11) The work system described in Supplementary Note 1 may further include a formation determination unit (240) that determines a formation consisting of multiple equipment operators who will operate the work equipment and multiple mobile body operators who will operate the work mobile body, based on work information, which is information related to the work, equipment operator candidate data related to an applicant who wishes to operate the work equipment, and mobile body operator candidate data related to an applicant who wishes to operate the work mobile body. This configuration increases the motivation of the equipment operators and mobile body operators for a specified work. Therefore, a better work system can be provided.

[0114] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments. [Explanation of symbols]

[0115] 10...Work system 12...Work equipment 14...Working vehicle 16...Management device 18...Remote control device for equipment 20...Remote control device for mobile objects 22...Communication line 24, 26...Imaging device 40: Detection unit 42: Drive unit 44, 80, 124, 164...Communication section 46, 82, 126, 166...Calculation section 48, 84, 128, 168...Memory unit 60...Mobile unit control unit 62...equipment control unit 64...image acquisition unit 100... Mobile object operation signal acquisition unit 102... Equipment operation signal acquisition unit 104...mobile information relay unit 106...image recognition unit 108...image generating unit 120, 160...display unit 122, 162... Operation detection unit 140, 182... Image signal acquisition unit 142, 184...Display control unit 144...Equipment remote control unit 180...Movement information acquisition unit 182...Image signal acquisition unit 186...mobile body remote control unit 200...work information acquisition unit 202...Operator determination unit 210...Data of person who wishes to operate equipment 212...Mobile object operator applicant data 220...Equipment operator determination unit 222... Mobile operator determination unit 240... Formation determination unit

Claims

1. A work system in which work can be performed by remotely operated work equipment, an image generation unit that generates a virtual space image corresponding to the real space around the working vehicle equipped with the work equipment; a display control unit that displays the virtual space image generated by the image generation unit on a display unit; Equipped with The image generation unit generates the virtual space image including a virtual image corresponding to the work equipment.

2. 2. The work system according to claim 1, The image generation unit generates the virtual space image in which image processing has been applied to a part corresponding to a work object that is the subject of the work.

3. 3. The work system according to claim 2, A work system, wherein the image processing is image processing that emphasizes the work object.

4. 3. The work system according to claim 2, A work system, wherein the image processing is image processing that superimposes an image corresponding to the work object on the work object.

5. 2. The work system according to claim 1, The working vehicle is remotely controlled.

6. 2. The work system according to claim 1, The work system further includes an operator determination unit that determines an equipment operator who will operate the work equipment and a mobile body operator who will operate the work mobile body based on work information, which is information related to the work, equipment operator candidate data related to an equipment operator who wishes to operate the work equipment, and mobile body operator candidate data related to a mobile body operator who wishes to operate the work mobile body.

7. 2. The work system according to claim 1, further comprising an equipment operator determination unit that determines an equipment operator who will operate the work equipment based on work information, which is information about the work, and equipment operator candidate data related to an equipment operator candidate who wishes to operate the work equipment; A work system in which the equipment operator desire data includes at least one of information regarding the work equipment that the equipment operator desires to operate, the work history of the equipment operator desires, and the skills of the equipment operator desires to operate.

8. 2. The work system according to claim 1, a mobile body operator determination unit that determines a mobile body operator who will operate the work mobile body based on work information, which is information related to the work, and mobile body operator candidate data related to a mobile body operator who wishes to operate the work mobile body; A work system in which the mobile object operator data includes at least one of information regarding the work mobile object that can be operated by the mobile object operator, the mobile object operation history of the mobile object operator, and the skills of the mobile object operator.

9. 2. The work system according to claim 1, further comprising an equipment operator determination unit that determines an equipment operator who will operate the work equipment based on work information, which is information about the work, and equipment operator candidate data regarding an equipment operator candidate who wishes to operate the work equipment, A work system, wherein the equipment operator applicant data includes at least one of information regarding a work area that the equipment operator applicant can handle and information regarding geographical conditions that the equipment operator applicant can handle.

10. 2. The work system according to claim 1, a mobile body operator determination unit that determines a mobile body operator who will operate the work mobile body based on work information, which is information related to the work, and mobile body operator candidate data related to a mobile body operator who wishes to operate the work mobile body; A work system in which the mobile object operator desire data includes at least one of information regarding a work area that the mobile object operator desires to handle and information regarding geographical conditions that the mobile object operator desires to handle.

11. 2. The work system according to claim 1, The work system further includes a formation determination unit that determines a formation consisting of a plurality of equipment operators who will operate the work equipment and a plurality of mobile body operators who will operate the work mobile body, based on work information, which is information related to the work, equipment operator candidate data related to equipment operator candidates who wish to operate the work equipment, and mobile body operator candidate data related to mobile body operators who wish to operate the work mobile body.

Images