Work system

Abstract

To provide an excellent work system.SOLUTION: A work system 10 where a work mobile body 12 to be operated by a user U performs a work includes: an image acquisition section for acquiring a close-up image in the periphery of the work mobile body 12; and a head-mounted display 14 mounted on the user U so as to provide a provision image corresponding to the close-up image acquired by the image acquisition section to the user U.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a work system. [Background technology]

[0002] Patent Document 1 discloses a virtual space display system. In this system, a passenger in an automatically driven vehicle wears a head-mounted display. The head-mounted display displays a virtual space image that differs from the actual environment around the vehicle to the passenger in the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6232649 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Patent Document 1 merely discloses the display of a virtual space that is different from the real environment. Recently, there has been a demand for providing a good working system.

[0005] The present invention aims to solve the above-mentioned problems. [Means for solving the problem]

[0006] An aspect of the present disclosure is a work system in which work can be performed by a work vehicle operated by a user, comprising an image acquisition unit that acquires a close-up image of the surroundings of the work vehicle, and a head-mounted display that is worn by the user and provides the user with an image corresponding to the close-up image acquired by the image acquisition unit. [Effects of the Invention]

[0007] According to the present invention, a good working system can be provided. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a work system and work performed by a working vehicle. [Figure 2] 2A and 2B are functional block diagrams of a working vehicle and a head-mounted display, respectively. [Figure 3] FIG. 3 is a diagram illustrating an example of an image provided to a user by a head-mounted display. [Figure 4] FIG. 4 is a flowchart showing an example of the operation of the working vehicle. [Figure 5] FIG. 5 is a flowchart showing an example of the operation of the head-mounted display. [Figure 6] FIG. 6 is a functional block diagram of the working vehicle. [Figure 7] FIG. 7 is a flowchart showing an example of the operation of the working vehicle. [Figure 8] FIG. 8 is a diagram illustrating an example of work performed by a working vehicle. [Figure 9] FIG. 9 is a functional block diagram of the working vehicle. [Figure 10] 10A and 10B are diagrams illustrating examples of images provided to the user by the head-mounted display. [Figure 11] FIG. 11 is a flowchart showing an example of the operation of the working vehicle. [Figure 12] FIG. 12 is a diagram illustrating an example of work performed by a working vehicle. [Figure 13] FIG. 13 is a functional block diagram of the working vehicle. [Figure 14] 14A and 14B are diagrams illustrating examples of images provided to the user by the head-mounted display. [Figure 15] FIG. 15 is a flowchart showing an example of the operation of the working vehicle. [Figure 16]FIG. 16 is a diagram illustrating an example of work performed by a working vehicle. [Figure 17] FIG. 17 is a functional block diagram of the working vehicle. [Figure 18] FIG. 18 is a flowchart showing an example of the operation of the working vehicle. [Figure 19] FIG. 19 is a diagram illustrating an example of work performed by a working vehicle. [Figure 20] FIG. 20 is a functional block diagram of the working vehicle. [Figure 21] FIG. 21 is a diagram illustrating an example of an image provided to the user by the head-mounted display. [Figure 22] FIG. 22 is a flowchart showing another example of the operation of the working vehicle. [Figure 23] FIG. 23 is a flowchart showing another example of the operation of the head mounted display. [Figure 24] FIG. 24 is a diagram showing the functional blocks of a working vehicle and a server for acoustic data. DETAILED DESCRIPTION OF THE INVENTION

[0009] A working system according to one embodiment will be described with reference to the drawings. Fig. 1 is a diagram illustrating a working system 10 and work performed by a working vehicle 12. The working system 10 includes a working vehicle 12 and a head-mounted display 14.

[0010] The work vehicle 12 performs work such as mowing while moving. In the example shown in FIG. 1 , the work vehicle 12 moves within land L along a predetermined movement route Rt. As the work vehicle 12 moves within land L, it mows grass G growing on the land L. Note that in this embodiment, the work vehicle 12 performs mowing work, but the work is not limited to this. The work may be, for example, farm work, cleaning work, civil engineering work, transportation work, etc.

[0011] The work vehicle 12 is directly operated by a user U. In this embodiment, the work vehicle 12 is described as being directly operated by a user U on board the work vehicle 12, but the work vehicle 12 may also be remotely operated by a user U not on board the work vehicle 12. The user U may wear a head-mounted display 14. The head-mounted display 14 provides the user U with a provided image, which will be described later. In addition, if the work vehicle 12 is equipped with a windshield display and an audio output device, the windshield display and the audio output device may be used instead of the head-mounted display 14.

[0012] The working vehicle 12 is equipped with an imaging device 16. The imaging device 16 may be configured with a camera. The imaging device 16 acquires close-up images of the surroundings of the working vehicle 12. More specifically, the imaging device 16 acquires close-up images of subjects located around the working vehicle 12. The imaging device 16 may be equipped with, for example, a lens for acquiring close-up images. Such a lens may include, but is not limited to, a magnifying lens such as a macro lens (close-up lens). Using a magnifying lens makes it possible to magnify and photograph small subjects.

[0013] The imaging device 16 obtains close-up images by magnifying and capturing images of subjects around the working vehicle 12. In the close-up images thus obtained, even relatively small subjects are captured with high definition.

[0014] Note that a close-up image may be obtained by enlarging an image obtained without using a macro lens. Furthermore, the imaging device 16 may obtain both a close-up image and an image with a smaller magnification than the close-up image. For example, the imaging device 16 may be provided with a first imaging device for obtaining a close-up image and a second imaging device for obtaining an image with a smaller magnification than the close-up image.

[0015] As will be described later, the image generated by the imaging device 16 is output to the work vehicle 12. Note that when a user U wearing a head-mounted display 14 boards the work vehicle 12, the imaging device 16 may be provided in the head-mounted display 14.

[0016] The close-up image acquired by the imaging device 16 may be an image captured at a low angle. For example, capturing an image of grass G at a low angle creates an impactful image of the grass G towering tall. As will be described later, the work vehicle 12 generates an image to be provided that has an impact corresponding to the close-up image.

[0017] An image signal indicating the generated provided image is provided from the work vehicle 12 to the head-mounted display 14 via wireless communication. The provided image is provided to the user U by the head-mounted display 14. The head-mounted display 14 may provide at least a portion of the provided image depending on the direction of the user U's face. By viewing the powerful provided image through the head-mounted display 14, the user U can feel a more extraordinary sense of entertainment than when viewing the grass G at a normal size. Therefore, the user U can enjoy mowing the grass G using the work vehicle 12.

[0018] 2A is a functional block diagram of the work vehicle 12. The work vehicle 12 has a detection unit 20, a drive unit 22, a communication unit 24, a calculation unit 26, and a storage unit 28. The detection unit 20 detects behavior related to the movement of the work vehicle 12. The detection unit 20 outputs behavior information based on the detection results of the behavior to the calculation unit 26. The detection unit 20 is configured with, for example, an encoder, an acceleration sensor, a gyro sensor, a positioning sensor, etc.

[0019] The drive unit 22 may be configured, for example, with a battery, an electric motor, an internal combustion engine, a power transmission mechanism, left and right wheels, etc. The power transmission mechanism transmits power from the electric motor or the internal combustion engine to the left and right wheels. The communication unit 24 may be configured, for example, with a wireless communication module equipped with an antenna, etc. The communication unit 24 may transmit signals to the outside of the work vehicle 12. The communication unit 24 may also receive signals from the outside of the work vehicle 12.

[0020] The calculation unit 26 may be configured by a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). That is, the calculation unit 26 may be configured by processing circuitry. At least a part of the calculation unit 26 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array). At least a part of the calculation unit 26 may be realized by an electronic circuit including discrete devices.

[0021] The storage unit 28 is a computer-readable storage medium. The storage unit 28 is configured with a volatile memory (not shown) and a non-volatile memory (not shown). The volatile memory is, for example, a RAM (Random Access Memory). The volatile memory is used as a working memory for the processor.

[0022] The nonvolatile memory is, for example, a ROM (Read Only Memory), a flash memory, etc. The nonvolatile memory stores programs executed by the processor, image data, acoustic data, travel route information, etc. In other words, the working vehicle 12 has a program product of the programs executed by the processor. At least a portion of the storage unit 28 may be provided in the above-mentioned processor, integrated circuit, etc.

[0023] The calculation unit 26 has a moving object control unit 40, an image acquisition unit 42, an image recognition unit 44, a movement direction determination unit 46, an image processing unit 48, and a sound generation unit 50. When the calculation unit 26 executes a program stored in the storage unit 28, the moving object control unit 40, the image acquisition unit 42, the image recognition unit 44, the movement direction determination unit 46, the image processing unit 48, and the sound generation unit 50 can be realized.

[0024] The mobile unit control unit 40 controls operations related to the movement of the work mobile unit 12 in response to operation of the work mobile unit 12 by the user U. The mobile unit control unit 40 can also autonomously control operations related to the movement of the work mobile unit 12. The image acquisition unit 42 acquires close-up images of the surroundings of the work mobile unit 12. If the close-up images include a specific subject image, the image recognition unit 44 recognizes the subject image. The subject image is recognized using image data stored in the memory unit 28, for example.

[0025] The movement direction determination unit 46 acquires a positioning signal from the detection unit 20. The movement direction determination unit 46 detects the current position and current orientation of the work mobile unit 12 based on the acquired positioning signal. The movement direction determination unit 46 acquires movement route information related to a predetermined movement route Rt from the storage unit 28. The movement direction determination unit 46 determines a movement direction corresponding to the movement route Rt at the current position of the work mobile unit 12 based on the current position and orientation of the work mobile unit 12 and the predetermined movement route Rt.

[0026] The image processing unit 48 may perform image processing on the close-up image. The image processing is performed, for example, using image data stored in the storage unit 28. The image processing unit 48 performs image processing to emphasize the image of the subject recognized by the image recognition unit 44. For example, the image of the subject is displayed enlarged. The image of the subject may be displayed surrounded by a closed figure. A figure may be placed around the image of the subject to make it stand out. The image processing unit 48 performs image processing to indicate the direction of movement determined by the movement direction determination unit 46. An arrow figure indicating the direction of movement is displayed.

[0027] In this way, the image processing unit 48 generates a presentation image corresponding to the close-up image. The image processing unit 48 transmits an image signal indicating the generated presentation image to the head-mounted display 14 via the communication unit 24. The sound generation unit 50 generates a sound signal indicating sound such as a sound effect, for example, using sound data stored in the storage unit 28. The sound generation unit 50 transmits the generated sound signal to the head-mounted display 14 via the communication unit 24.

[0028] 2B is a functional block diagram of the head-mounted display 14. The head-mounted display 14 has a display unit 70, an audio output unit 72, a communication unit 74, a calculation unit 76, and a storage unit 78. The display unit 70 displays a provided image in response to an image signal transmitted from the work vehicle 12. In this way, the display unit 70 can provide the provided image to the user U. The audio output unit 72 outputs audio in response to the audio signal transmitted from the work vehicle 12. In this way, the audio output unit 72 can provide audio, such as sound effects, to the user U.

[0029] The communication unit 74 may be configured by, for example, a wireless communication module including an antenna, etc. The communication unit 74 may receive a signal from outside the head mounted display 14.

[0030] The calculation unit 76 may be configured by a processor such as a CPU or a GPU. That is, the calculation unit 76 may be configured by a processing circuit. At least a part of the calculation unit 76 may be realized by an integrated circuit such as an ASIC or an FPGA. At least a part of the calculation unit 76 may be realized by an electronic circuit including discrete devices.

[0031] The memory unit 78 is a computer-readable storage medium. The memory unit 78 is composed of a volatile memory (not shown) and a non-volatile memory (not shown). The volatile memory is, for example, a RAM. The volatile memory is used as a working memory for the processor. The non-volatile memory is, for example, a ROM, a flash memory, etc. The non-volatile memory stores programs executed by the processor and other necessary data. In other words, the head-mounted display 14 has a program product of the programs executed by the processor. At least a part of the memory unit 78 may be provided in the above-mentioned processor, integrated circuit, etc.

[0032] The calculation unit 76 has a signal acquisition unit 90, a display control unit 92, and an audio control unit 94. The calculation unit 76 executes a program stored in the storage unit 78, thereby realizing the signal acquisition unit 90, the display control unit 92, and the audio control unit 94.

[0033] The signal acquisition unit 90 acquires an image signal representing a provided image from the image processing unit 48 of the working vehicle 12 via the communication unit 74. The signal acquisition unit 90 acquires an audio signal representing audio from the audio generation unit 50 of the working vehicle 12 via the communication unit 74. The display control unit 92 causes the display unit 70 to display a provided image corresponding to the image signal acquired by the signal acquisition unit 90. The audio control unit 94 causes the audio output unit 72 to output audio corresponding to the audio signal acquired by the signal acquisition unit 90.

[0034] FIG. 3 is a diagram illustrating an example of a provided image provided to the user U by the head-mounted display 14. The close-up image acquired by the image acquisition unit 42 of the work vehicle 12 includes a specific subject image Ti along with an image of grass G, which appears towering and impressive. The image processing unit 48 of the work vehicle 12 performs image processing on the close-up image, superimposing the images Ad, Hr, and Hf on the close-up image. In this way, the provided image shown in FIG. 3 is obtained, which has an impressive appearance corresponding to the close-up image. This makes it possible to provide an excellent work system 10.

[0035] The image Ad indicates the movement direction of the work vehicle 12 according to the predetermined movement route Rt. In the example shown in Fig. 3, an arrow graphic indicating the movement direction is displayed as the image Ad. By superimposing the image Ad on the close-up image, the user U can easily operate the work vehicle 12.

[0036] In the example shown in FIG. 3, the subject image Ti itself is enlarged and displayed, thereby emphasizing the subject image Ti. Images Hr and Hf further emphasize the subject image Ti. Image Hr displays a closed annular figure surrounding the subject image Ti. Image Hf displays a figure prominently arranged around the subject image Ti. By superimposing images Hr and Hf on the close-up image, the user U can enjoy working using the work vehicle 12.

[0037] Fig. 4 is a flowchart showing an example of the operation of the work vehicle 12. Fig. 4 shows the processing procedure when the work vehicle 12 generates an image to be provided. This processing procedure is performed by the calculation unit 26 of the work vehicle 12 executing a program stored in the storage unit 28. This processing procedure is repeatedly performed while the work vehicle 12 is moving and working.

[0038] When this processing procedure starts, in step S1, the image acquisition unit 42 acquires a close-up image of the surroundings of the working vehicle 12. In step S2, the image recognition unit 44 determines whether or not a specific subject image Ti is included in the close-up image acquired in step S1. If the answer is YES in step S2, this processing procedure proceeds to step S3. If the answer is NO in step S2, this processing procedure proceeds to step S4.

[0039] In step S3, the image processing unit 48 performs image processing on the close-up image to enhance the subject image Ti recognized in step S3. In step S4, the movement direction determination unit 46 detects the current position and orientation of the work vehicle 12. In step S5, the movement direction determination unit 46 acquires movement path information from the storage unit 28.

[0040] In step S6, the image processing unit 48 performs image processing on the close-up image. For example, the images Ad, Hr, and Hf shown in Fig. 3 are superimposed on the close-up image. In this way, the image processing unit 48 generates an image signal representing a provided image corresponding to the close-up image.

[0041] In step S7, the sound generation unit 50 generates a sound signal indicating sound such as a sound effect. In step S8, the image processing unit 48 transmits the image signal generated in step S6 to the head-mounted display 14 via the communication unit 24. The sound generation unit 50 transmits the sound signal generated in step S7 to the head-mounted display 14 via the communication unit 24. When the processing of step S8 is completed, this processing procedure ends.

[0042] This processing procedure may be performed by a management device capable of communicating with the working vehicle 12 via a network.

[0043] Fig. 5 is a flowchart showing an example of the operation of the head-mounted display 14. Fig. 5 shows the processing procedure when the head-mounted display 14 provides a provided image to the user U. This processing procedure is performed by the calculation unit 76 of the head-mounted display 14 by executing a program stored in the storage unit 78. This processing procedure is repeatedly performed while the working vehicle 12 is moving and working.

[0044] When this processing procedure starts, in step S21, the signal acquisition unit 90 acquires an image signal and an audio signal from the work vehicle 12. The image signal represents a provided image generated by the image processing unit 48 of the work vehicle 12. The audio signal represents audio generated by the audio generation unit 50 of the work vehicle 12. In step S22, the display control unit 92 causes the display unit 70 to display a provided image corresponding to the image signal acquired in step S21. In step S23, the audio control unit 94 causes the audio output unit 72 to output audio corresponding to the audio signal acquired in step S21. When the processing of step S23 is completed, this processing procedure ends.

[0045] The above-described embodiment may be modified as follows: In the following modifications, explanations that overlap with the above-described embodiment will be omitted.

[0046] (Variation 1) In the above-described embodiment, the sound generation unit 50 generates a sound signal representing sound such as a sound effect, but the sound may change depending on the work status of the work vehicle 12. In this first modification, the sound generation unit 50 changes the sound signal depending on the work status of the work vehicle 12.

[0047] FIG. 6 is a functional block diagram of the working vehicle 12. In FIG. 6, the same reference numerals are used for components common to FIG. 2A. A description of these components will be omitted. The calculation unit 26 shown in FIG. 6 further includes a determination unit 110. The determination unit 110 can be realized by the calculation unit 26 executing a program stored in the storage unit 28.

[0048] The determination unit 110 determines the work status of the work mobile unit 12. The determination unit 110 determines, for example, the movement speed of the work mobile unit 12 as the work status of the work mobile unit 12. In this case, the determination unit 110 determines the movement speed of the work mobile unit 12 based on information related to the movement speed of the work mobile unit 12. The information related to the movement speed of the work mobile unit 12 is included in the behavior information output by the detection unit 20. Furthermore, the determination unit 110 compares the movement speed of the work mobile unit 12 with a speed threshold value stored in advance in the memory unit 28.

[0049] If the movement speed of the work mobile unit 12 is equal to or greater than the speed threshold, the determination unit 110 determines that the work mobile unit 12 is moving smoothly and the work situation is good. In this case, it is considered that the work of the work mobile unit 12 is not behind schedule. On the other hand, if the movement speed is less than the speed threshold, the determination unit 110 determines that the work mobile unit 12 is moving at a low speed and the work situation is poor. In this case, it is considered that the work of the work mobile unit 12 is behind schedule.

[0050] The work status may be determined based on the work time of the work mobile unit 12 instead of the movement speed of the work mobile unit 12. In this case, the determination unit 110 compares the work time in a predetermined section of the predetermined movement route Rt with a time threshold value pre-stored in the memory unit 28. If the work time is equal to or greater than the time threshold value, the determination unit 110 determines the work status to be good. If the work time is less than the time threshold value, the determination unit 110 determines the work status to be poor.

[0051] The sound generator 50 changes the generated sound signal depending on the work situation determined by the determiner 110. When the determiner 110 determines that the work situation is poor, the sound generator 50 generates a sound signal that indicates sound with a higher beat rate than when the work situation is determined to be good. This can encourage the user U to improve the work situation. In other words, the work performed by the work vehicle 12 can be sped up.

[0052] FIG. 7 is a flowchart showing an example of the operation of the work vehicle 12. FIG. 7 shows the processing procedure when the work vehicle 12 generates an image to be provided. This processing procedure is performed by the calculation unit 26 of the work vehicle 12 executing a program stored in the storage unit 28. This processing procedure is repeated while the work vehicle 12 is moving and working. Steps similar to those described above with reference to FIG. 4 are given the same reference numerals, and explanations will be omitted where appropriate.

[0053] When the processing of step S6 is completed, the process proceeds to step S31. In step S31, the determination unit 110 acquires information related to the movement speed of the work vehicle 12 from the detection unit 20. The information related to the movement speed of the work vehicle 12 is included in the behavior information output by the detection unit 20. In step S32, the determination unit 110 determines the work status of the work vehicle 12 based on the information acquired in step S31.

[0054] In step S33, the sound generator 50 generates a sound signal according to the work situation determined in step S32. When the process of step S33 is completed, the process proceeds to step S8.

[0055] (Variation 2) In the embodiment described above, the work vehicle 12 cuts grass G growing on the land L while moving within the land L, but the land L may also include a work area and a non-work area. In this second modification, the land L includes a work area and a non-work area. A work area is an area that is the target of work by the work vehicle 12. A non-work area is an area that is not the target of work by the work vehicle 12.

[0056] FIG. 8 is a diagram illustrating work performed by a work vehicle 12. In the example shown in FIG. 8, a work area Lt and a non-work area Ln are adjacent to each other. The work vehicle 12 is performing work in the work area Lt while moving along a predetermined movement route Rt. The work vehicle 12 is about to arrive at the end point of the movement route Rt. The non-work area Ln is located immediately to the right of the moving work vehicle 12. In this case, a provided image is provided to the user U in which a close-up image has been subjected to image processing that distinguishes between the work area Lt and the non-work area Ln.

[0057] Figure 9 is a functional block diagram of the working vehicle 12. In Figure 9, the same reference numerals are used for components that are common to Figure 2A. A description of these components will be omitted. The calculation unit 26 shown in Figure 9 further includes a work area determination unit 120. The work area determination unit 120 can be realized by the calculation unit 26 executing a program stored in the storage unit 28.

[0058] Information relating to the work area Lt and the non-work area Ln is stored in advance in the memory unit 28. The image recognition unit 44 acquires information relating to the current position of the work vehicle 12, the current orientation of the work vehicle 12, and the predetermined travel route Rt from the travel direction determination unit 46. The image recognition unit 44 can recognize an image of the work area Lt and an image of the non-work area Ln in the close-up image based on the information relating to the work area Lt and the non-work area Ln, the current position of the work vehicle 12, the current orientation of the work vehicle 12, and the predetermined travel route Rt.

[0059] The image processing unit 48 can perform image processing on the close-up image. The image processing unit 48 performs image processing on the close-up image to distinguish between the image of the work area Lt recognized by the image recognition unit 44 and the image of the non-work area Ln. The image processing unit 48 generates a provided image that has been subjected to this image processing. By providing the provided image to the user U, the user U can be prevented from accidentally having the work vehicle 12 work in the non-work area Ln.

[0060] 10A and 10B are diagrams illustrating examples of provided images provided to the user U by the head-mounted display 14. As shown in FIG. 8, when a non-work area Ln is located immediately to the right of the work vehicle 12, the provided images shown in FIGS. 10A and 10B include an image Mt of the work area Lt and an image Mn of the non-work area Ln. The provided images are obtained by performing image processing by the image processing unit 48 to distinguish between the image Mt of the work area Lt and the image Mn of the non-work area Ln.

[0061] In the provided image shown in Fig. 10A, a close-up image is used for the image Mt of the area to be worked Lt, while an image with a smaller magnification than the close-up image is used for the image Mn of the area not to be worked Ln. In the provided image shown in Fig. 10B, a close-up image is used as is for the image Mt of the area to be worked Lt, while a processed image in which the close-up image is processed with a semi-transparent mask is used for the image Mn of the area not to be worked Ln. Instead of a semi-transparent mask, a processed image processed by filling in with a single color may be used.

[0062] FIG. 11 is a flowchart showing an example of the operation of the work vehicle 12. FIG. 11 shows the processing procedure when the work vehicle 12 generates an image to be provided. This processing procedure is performed by the calculation unit 26 of the work vehicle 12 executing a program stored in the storage unit 28. This processing procedure is repeated while the work vehicle 12 is moving and working. Steps similar to those described above with reference to FIG. 4 are given the same reference numerals, and explanations will be omitted where appropriate.

[0063] Once the processing of step S6 is complete, the process proceeds to step S41. In step S41, the image recognition unit 44 recognizes, in the close-up image, an image Mt of the work area Lt and an image Mn of the non-work area Ln. In step S42, the image processing unit 48 performs image processing on the close-up image to distinguish between the image Mt of the work area Lt recognized in step S41 and the image Mn of the non-work area Ln. In this way, the image processing unit 48 generates an image signal indicating a provided image corresponding to the close-up image. Once the processing of step S42 is complete, the process proceeds to step S7.

[0064] (Variation 3) In the above-described embodiment, the work vehicle 12 cuts grass G growing on the land L while moving within the land L, but the land L may include a non-recommended area. In this third modification, the land L includes a non-recommended area. A non-recommended area is an area where it is not recommended for the work vehicle 12 to enter. Examples of non-recommended areas include areas with steep slopes, areas with mud, and areas with large rocks scattered around.

[0065] FIG. 12 is a diagram illustrating work performed by a work mobile unit 12. In the example shown in FIG. 12, a portion of land L is an unrecommended area Lu. The work mobile unit 12 is performing work in a work target area Lt while moving along a predetermined movement route Rt. The work mobile unit 12 is about to arrive at the end point of the movement route Rt. The unrecommended area Lu is located immediately to the right of the moving work mobile unit 12. In this case, a provided image is provided to the user U, in which image processing has been applied to a close-up image to attract the user U's attention. The image processing is applied to a portion of the close-up image that corresponds to the unrecommended area Lu.

[0066] Fig. 13 is a functional block diagram of the working vehicle 12. In Fig. 13, the same reference numerals are used for components common to Fig. 2A. A description of these components will be omitted. The calculation unit 26 shown in Fig. 13 further includes a non-recommended area determination unit 130. The non-recommended area determination unit 130 can be realized by the calculation unit 26 executing a program stored in the storage unit 28.

[0067] Information about the non-recommended area Lu is stored in advance in the storage unit 28. The image recognition unit 44 acquires information about the current position of the work mobile unit 12, the current orientation of the work mobile unit 12, and the predetermined travel route Rt from the travel direction determination unit 46. The image recognition unit 44 can recognize an image of a portion corresponding to the non-recommended area Lu in the close-up image based on the information about the non-recommended area Lu, the current position of the work mobile unit 12, the current orientation of the work mobile unit 12, and the predetermined travel route Rt.

[0068] The image processing unit 48 can perform image processing on the close-up image. The image processing unit 48 performs image processing on the close-up image to draw the user U's attention to the portion corresponding to the unrecommended area Lu recognized by the image recognition unit 44. The image processing unit 48 generates a provided image that has been subjected to such image processing. By providing the provided image to the user U, the user U can be prevented from accidentally causing the work vehicle 12 to enter the unrecommended area Lu.

[0069] 14A and 14B are diagrams illustrating examples of images provided by the head-mounted display 14 to the user U. As shown in Fig. 12, when a non-recommended area Lu is located immediately to the right of the work vehicle 12, the images provided in Fig. 14A and 14B include a region Pu corresponding to the non-recommended area Lu. The image processing unit 48 performs image processing on the region Pu corresponding to the non-recommended area Lu to draw the user U's attention to the image provided.

[0070] In the provided image shown in FIG. 14A, a scary image Mh1 that creates a sense of tension in the user U is superimposed on a region Pu that corresponds to the non-recommended region Lu. In FIG. 14A, an image showing blood is used as an example of the scary image Mh1. This is expected to attract the user U's attention. In the provided image shown in FIG. 14B, a scary image Mh2 that creates a sense of tension in the user U is superimposed on a region Pu that corresponds to the non-recommended region Lu. In FIG. 14B, an image showing a monster is used as an example of the scary image Mh2. This is expected to attract the user U's attention.

[0071] FIG. 15 is a flowchart showing an example of the operation of the work vehicle 12. FIG. 15 shows the processing procedure when the work vehicle 12 generates an image to be provided. This processing procedure is performed by the calculation unit 26 of the work vehicle 12 executing a program stored in the storage unit 28. This processing procedure is repeated while the work vehicle 12 is moving and working. Steps similar to those described above using FIG. 4 are given the same reference numerals, and explanations will be omitted where appropriate.

[0072] When the processing of step S6 is completed, the process proceeds to step S51. In step S51, the image recognition unit 44 recognizes an image of the part Pu corresponding to the non-recommended area Lu in the close-up image. In step S52, the image processing unit 48 performs image processing on the close-up image to draw the user U's attention to the part Pu corresponding to the non-recommended area Lu recognized in step S41. In this way, the image processing unit 48 generates an image signal indicating a provided image corresponding to the close-up image. When the processing of step S52 is completed, the process proceeds to step S7.

[0073] (Variation 4) To enhance the game aspect of the work system 10, points may be awarded to the user U. In this fourth modification, points are awarded to the user U according to the work performed by the work mobile unit 12 based on the operation of the work mobile unit 12 by the user U. Specifically, points are determined based on the work history of the work performed by the work mobile unit 12. The work history is, for example, a total distance obtained by adding up the distance traveled by the user U with the work mobile unit 12 while working. In this fourth modification, the total distance is used as the work history, but a total time obtained by adding up the work time of the work performed by the work mobile unit 12 may also be used.

[0074] FIG. 16 is a diagram illustrating work performed by a work vehicle 12. In the example shown in FIG. 16, two work vehicles 12 are mowing grass G on the same land L. One of the two work vehicles 12 has moved a distance D1 from the position where work started to its current position. The other of the two work vehicles 12 has moved a distance D2 from the position where work started to its current position. If the distance D1 of one work vehicle 12 is greater than the distance D2 of the other work vehicle 12, the points awarded to the user U of one work vehicle 12 will be greater than the points awarded to the user U of the other work vehicle 12.

[0075] FIG. 17 is a functional block diagram of the working vehicle 12. In FIG. 17, the same reference numerals are used for components common to FIG. 2A. A description of these components will be omitted. The calculation unit 26 shown in FIG. 17 further includes a history management unit 140 and a point assignment unit 142. The calculation unit 26 executes a program stored in the storage unit 28, thereby realizing the history management unit 140 and the point assignment unit 142.

[0076] The history management unit 140 acquires information about the work history of work performed by the work mobile unit 12 based on the operation of the work mobile unit 12 by the user U. The history management unit 140 manages the above-mentioned total distance as the work history. The total distance traveled by the work mobile unit 12 is obtained based on information about the current position of the work mobile unit 12 detected by the movement direction determination unit 46 and information about the movement path of the work mobile unit 12 acquired by the movement direction determination unit 46. In this way, the history management unit 140 acquires information about the total distance. The history management unit 140 stores the total distance in the memory unit 28.

[0077] The point assigning unit 142 assigns points to the user U according to the total distance stored in the storage unit 28. For example, the point assigning unit 142 assigns a predetermined amount of points to the user U each time the total distance increases by a certain distance. The point assigning unit 142 stores the total value of the points assigned to the user U in the storage unit 28. This motivates the user U to operate the work vehicle 12 and perform work.

[0078] FIG. 18 is a flowchart showing an example of the operation of the work vehicle 12. FIG. 18 shows the processing procedure when the work vehicle 12 generates an image to be provided. This processing procedure is performed by the calculation unit 26 of the work vehicle 12 executing a program stored in the storage unit 28. This processing procedure is repeated while the work vehicle 12 is moving and working. Steps similar to those described above using FIG. 4 are given the same reference numerals, and explanations will be omitted where appropriate.

[0079] When the processing of step S6 is completed, the process proceeds to step S61. In step S61, the history management unit 140 acquires information about the total distance traveled by the work vehicle 12 as information about the work history of the work performed by the work vehicle 12 based on the operation of the work vehicle 12 by the user U. In step S62, the point assignment unit 142 assigns points to the user U according to the increase in the total distance. In step S63, the point assignment unit 142 saves the total value of the points assigned to the user U in the memory unit 28. When the processing of step S63 is completed, the process proceeds to step S7.

[0080] This processing procedure may be performed by a management device capable of network communication with the work vehicles 12. In the example shown in Fig. 16, points are awarded to each user U of two work vehicles 12. The management device can display the points of each of the multiple users U on the head-mounted displays 14 worn by each user U. This can induce competition among the users U to earn points, further motivating the users U to operate the work vehicles 12 and perform work.

[0081] (Variation 5) There may be obstacles within the land L. In that case, the work vehicle 12 may come into contact with the obstacle. Obstacles include, for example, rocks, trees, buildings, animals, etc. Other work vehicles 12 may also be obstacles. For example, when multiple work vehicles 12 are performing work within the same land L, the remaining work vehicles 12 are obstacles to one of the work vehicles 12. In this fifth variation, the work vehicle 12 autonomously controls its movement-related operations to avoid contact with obstacles.

[0082] Fig. 19 is a diagram illustrating work by a work vehicle 12. In the example shown in Fig. 19, a work vehicle 12A is performing work to cut grass G along a travel path Rt on land L and is about to come into contact with another work vehicle 12B, which is an obstacle. When the work vehicle 12A predicts contact with the obstacle, movement control of the work vehicle 12A switches from control based on operation by a user U to autonomous control.

[0083] FIG. 20 is a functional block diagram of the work vehicle 12A. In FIG. 20, the same reference numerals are used for components common to FIG. 2A. A description of these components will be omitted. The work vehicle 12A shown in FIG. 20 further includes an external environment detection unit 150 and an airbag 152. The external environment detection unit 150 may be configured with an external environment sensor such as a radar or LiDAR (Laser Imaging, Detection, and Ranging). The external environment detection unit 150 can detect the distance from the work vehicle 12A to an obstacle. The external environment detection unit 150 outputs external environment information related to the detected distance to the calculation unit 26.

[0084] The airbag 152 may be configured by an airbag module. The airbag module includes an airbag body, an inflator that supplies gas to the airbag body, and a drive circuit that drives the inflator.

[0085] The calculation unit 26 further includes a prediction unit 160. The prediction unit 160 can be realized by the calculation unit 26 executing a program stored in the storage unit 28. The prediction unit 160 predicts whether the work vehicle 12A will come into contact with an obstacle. For example, the prediction unit 160 calculates the TTC (Time To Collision) until the work vehicle 12A comes into contact with the obstacle based on behavior information and external environment information. The prediction unit 160 predicts that the work vehicle 12A will come into contact with the obstacle if the TTC is below a predetermined first time threshold.

[0086] When the prediction unit 160 predicts that the working vehicle 12A will come into contact with an obstacle, the mobile unit control unit 40 switches the movement control of the working vehicle 12A. That is, the mobile unit control unit 40 autonomously controls the working vehicle 12A to avoid contact between the working vehicle 12A and the obstacle. For example, the mobile unit control unit 40 may change the traveling direction of the working vehicle 12A or may stop the working vehicle 12A. This can avoid contact. Therefore, damage to the working vehicle 12A can be prevented.

[0087] If the work vehicle 12A inevitably comes into contact with an obstacle even though the movement control of the work vehicle 12A has been switched to autonomous control, the airbag 152 is activated. The prediction unit 160 determines that contact between the work vehicle 12A and the obstacle is unavoidable when the TTC is below a predetermined second time threshold. The second time threshold is smaller than the first time threshold. If the prediction unit 160 determines that contact between the work vehicle 12A and the obstacle is unavoidable, the vehicle control unit 40 outputs a drive signal to the airbag 152.

[0088] The airbag 152 is activated in response to the drive signal, and the airbag body is deployed. At this time, image processing indicating that the airbag 152 has been activated is performed on the close-up image by the image processing unit 48. FIG. 21 is a diagram illustrating an example of a provided image provided to the user U by the head-mounted display 14. In the example shown in FIG. 21, the provided image is generated by superimposing an image Mb corresponding to the airbag 152 on the close-up image. The image Mb is displayed in a manner that makes the airbag body appear to bounce. This can mitigate the mental upset that the user U may experience when coming into contact with an obstacle.

[0089] Figure 22 is a flowchart showing another example of the operation of the work vehicle 12. Figure 22 illustrates an example of a processing procedure for predicting contact between the work vehicle 12 and an obstacle. This processing procedure is performed by the calculation unit 26 of the work vehicle 12 by executing a program stored in the memory unit 28. This processing procedure is repeatedly performed together with the processing procedure shown in Figure 4 while the work vehicle 12 is moving and working.

[0090] When this processing procedure starts, in step S71, the prediction unit 160 predicts whether or not the work vehicle 12A will come into contact with an obstacle. The prediction in step S71 is made based on the TTC and the first time threshold value described above. If the result in step S71 is YES, this processing procedure proceeds to step S72. If the result in step S71 is NO, this processing procedure ends.

[0091] In step S72, the mobile unit control unit 40 autonomously controls the work vehicle 12A to avoid contact between the work vehicle 12A and the obstacle. In step S73, the prediction unit 160 determines whether contact between the work vehicle 12A and the obstacle is avoidable. The determination in step S73 is made based on the TTC and the second time threshold value described above. If the result in step S73 is YES, the process proceeds to step S74. If the result in step S73 is NO, the process proceeds to step S81.

[0092] After the work vehicle 12A has avoided contact with the obstacle, in step S74 the vehicle control unit 40 cancels the autonomous control of the work vehicle 12A that was started in step S72. The user U can again operate the work vehicle 12A. When the processing of step S74 is completed, this processing procedure ends.

[0093] In step S81, the vehicle control unit 40 outputs a drive signal to the airbag 152 to activate the airbag 152. In step S82, the image processing unit 48 acquires a close-up image of the surroundings of the work vehicle 12A. In step S83, the image processing unit 48 performs image processing on the close-up image acquired in step S82 to indicate that the airbag 152 has been activated. For example, the image Mb shown in FIG. 21 is superimposed on the close-up image. In this way, the image processing unit 48 generates an image signal indicating a provided image corresponding to the close-up image.

[0094] In step S84, the image processing unit 48 transmits the image signal generated in step S6 to the head-mounted display 14 via the communication unit 24. When the processing in step S84 is completed, this processing procedure ends.

[0095] Fig. 23 is a flowchart showing another example of the operation of the head-mounted display 14. Fig. 23 shows the processing procedure when the head-mounted display 14 provides a provided image to the user U. This processing procedure is performed by the calculation unit 76 of the head-mounted display 14 by executing a program stored in the storage unit 78. This processing procedure is repeatedly performed together with the processing procedure shown in Fig. 5 while the working vehicle 12A is moving and working.

[0096] When this processing procedure starts, in step S101, the signal acquisition unit 90 acquires an image signal from the work vehicle 12A. The image signal represents a provided image generated by the image processing unit 48 of the work vehicle 12A. In step S102, the display control unit 92 causes the display unit 70 to display a provided image corresponding to the image signal acquired in step S101. When the processing of step S102 is completed, this processing procedure ends.

[0097] This processing procedure may be performed by a management device capable of network communication with the work vehicle 12. In the example shown in Fig. 19, the work vehicle 12B also has the functional blocks shown in Fig. 20 like the work vehicle 12A, and if the user U of the work vehicle 12B is wearing a head-mounted display 14, this processing procedure may also be performed in the work vehicle 12B.

[0098] The management device can cause this processing procedure to be performed only on the work vehicle 12 that is moving at a slower speed, between the work vehicle 12A and the work vehicle 12B. This further increases the possibility of avoiding contact between the work vehicle 12A and the work vehicle 12B.

[0099] (Variation 6) In the above-described embodiment, the sound generation unit 50 generates a sound signal representing sound such as a sound effect using sound data stored in the storage unit 28. The sound data may be stored in advance in the storage unit 28, but is not limited to this. The sound data may be acquired through network communication. In this sixth modification, the sound generation unit 50 generates a sound signal representing sound such as a sound effect using sound data acquired through network communication.

[0100] 24 is a diagram showing the functional blocks of the work vehicle 12 and the acoustic data server 170. The work vehicle 12 and the server 170 are connected to each other via a communication line 180 such as the Internet. The work vehicle 12 can obtain acoustic data used for the above-mentioned sounds from the server 170.

[0101] In the functional block diagram of the working vehicle 12 shown in Fig. 24, the same reference numerals are used for components common to Fig. 2A. Explanation of these components will be omitted. The calculation unit 26 shown in Fig. 24 further includes a data acquisition unit 190. The data acquisition unit 190 can be realized by the calculation unit 26 executing a program stored in the storage unit 28.

[0102] The data acquisition unit 190 can acquire acoustic data from the server 170 via the communication line 180 and the communication unit 24 at any time. The acoustic data can be used by the sound generation unit 50 when it generates an acoustic signal. The data acquisition unit 190 stores the acoustic data acquired through network communication in the storage unit 28. The sound generation unit 50 uses the acoustic data stored in the storage unit 28 to generate an acoustic signal representing sound such as a sound effect.

[0103] This allows the sound generation unit 50 to generate a variety of the latest sound signals using the sound data acquired at any time by the data acquisition unit 190. This provides the user U with an even greater motivation to operate the working vehicle 12 and perform work without getting bored.

[0104] (Variation 7) The above-described multiple modified examples may be combined as appropriate within a range that does not cause inconsistency.

[0105] The following additional notes are provided regarding the above-described embodiment and modifications.

[0106] (Appendix 1) A work system (10) in which work can be performed by a work vehicle (12) operated by a user (U) includes an image acquisition unit (42) that acquires a close-up image of the work vehicle's surroundings, and a head-mounted display (14) that is worn by the user and provides the user with a provided image corresponding to the close-up image acquired by the image acquisition unit. This configuration provides a provided image with impact corresponding to the close-up image. Therefore, a good work system can be provided.

[0107] (Appendix 2) The work system described in Supplementary Note 1 may further include an image processing unit (48) that can perform image processing on the close-up image, and the head-mounted display may provide the user with the provided image that has been subjected to the image processing that distinguishes between a work area (Lt) that is the area that is the target of the work and a non-work area (Ln) that is the area that is not the target of the work. With this configuration, it is possible to prevent the user from accidentally causing the work vehicle to work in a non-work area.

[0108] (Appendix 3) The work system described in Supplementary Note 1 may further include an image processing unit that performs image processing on the close-up image, and the head-mounted display may provide the user with the provided image that has been subjected to the image processing to draw the user's attention to a region (Pu) corresponding to a non-recommended area (Lu) where entry of the work vehicle is not recommended. This configuration can prevent the user from accidentally entering the work vehicle into a non-recommended area.

[0109] (Appendix 4) The work system described in Supplementary Note 1 may further include an image processing unit that performs image processing on the close-up image, and the head-mounted display may provide the user with the image that has been subjected to the image processing and indicates a movement direction according to a predetermined movement path (Rt). This configuration makes it easier for the user to operate the work vehicle.

[0110] (Appendix 5) The work system described in Supplementary Note 1 may further include an image processing unit that performs image processing on the close-up image, and when the close-up image includes a specific subject image (Ti), the head-mounted display may provide the user with the provided image that has been subjected to the image processing to emphasize the subject image. With this configuration, the user can enjoy working using the work vehicle.

[0111] (Appendix 6) The work system described in Supplementary Note 1 further includes a determination unit (110) that determines the work status of the work vehicle, a sound generation unit (50) that generates a sound signal, and a sound output unit (72) that provides the user with sound corresponding to the sound signal generated by the sound generation unit, and the sound generation unit may change the sound signal depending on the work status. This configuration can encourage the user to improve the work status. In other words, the work performed by the work vehicle can be sped up.

[0112] (Appendix 7) The work system described in Supplementary Note 1 may further include a point awarding unit (142) that awards points to the user according to the work performed by the work vehicle based on the user's operation of the work vehicle. This configuration motivates the user to operate the work vehicle and perform work.

[0113] (Appendix 8) The work system described in Supplementary Note 1 may further include a prediction unit (160) that predicts whether the work vehicle will come into contact with an obstacle, and a vehicle control unit (40) that controls the work vehicle to avoid contact between the work vehicle and the obstacle when the prediction unit predicts that the work vehicle will come into contact with the obstacle. With this configuration, contact between the work vehicle and the obstacle can be avoided, and therefore damage to the work vehicle can be prevented.

[0114] (Appendix 9) The work system described in Supplementary Note 8 may further include an image processing unit that performs image processing on the close-up image, wherein the mobile unit control unit activates an airbag (152) provided on the work mobile unit when contact between the work mobile unit and the obstacle cannot be avoided, and when the airbag is activated, the head-mounted display may provide the user with the provided image that has been subjected to the image processing, indicating that the airbag has been activated. With this configuration, the psychological upset that contact with an obstacle causes to the user can be alleviated.

[0115] (Appendix 10) The work system described in Supplementary Note 6 may further include a data acquisition unit (190) that acquires, via network communication, acoustic data that can be used to generate the acoustic signal, and the acoustic generation unit may generate the acoustic signal using the acoustic data acquired by the data acquisition unit. This configuration allows for the generation of a variety of up-to-date acoustic signals. This further motivates the user to operate the work vehicle and perform work without becoming bored.

[0116] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments. [Explanation of symbols]

[0117] 10...Work system 12...Work vehicle 14...Head-mounted display 16...Imaging device 24...Communication unit 26, 76...Calculation unit 28, 78...Memory unit 40...Mobile object control unit 42...Image acquisition unit 44...Image recognition unit 46...Movement direction determination unit 48...Image processing unit 50...Sound generation section 70...Display section 72...acoustic output unit 74...communication unit 90: Signal acquisition unit 92: Display control unit 94...acoustic control unit 110...determination unit 120...Work area determination unit 130...Non-recommended area determination unit 140... History management unit 142... Point assignment unit 150...External environment detection unit 152...Airbag 160...Prediction unit 170...Server 180...Communication line 190...Data acquisition unit

Claims

1. A work system in which work can be performed by a work vehicle operated by a user, an image acquisition unit that acquires a close-up image of the surroundings of the working vehicle; a head-mounted display that is worn by the user and provides the user with an image corresponding to the close-up image acquired by the image acquisition unit; A working system comprising:

2. 2. The work system according to claim 1, further comprising an image processing unit capable of performing image processing on the close-up image, The head-mounted display provides the user with the provided image that has been subjected to image processing to distinguish between a work area, which is the area that is the subject of the work, and a non-work area, which is the area that is not the subject of the work.

3. 2. The work system according to claim 1, further comprising an image processing unit capable of performing image processing on the close-up image, The head-mounted display provides the user with the provided image, which has been subjected to image processing to draw the user's attention to areas corresponding to non-recommended areas where entry of the work vehicle is not recommended.

4. 2. The work system according to claim 1, further comprising an image processing unit capable of performing image processing on the close-up image, The head-mounted display provides the user with the provided image that has been subjected to the image processing and that indicates a movement direction according to a predetermined movement path.

5. 2. The work system according to claim 1, further comprising an image processing unit capable of performing image processing on the close-up image, An operation system in which, if the close-up image includes an image of a specific subject, the head-mounted display provides the user with the provided image that has been subjected to the image processing to emphasize the image of the subject.

6. 2. The work system according to claim 1, a determination unit for determining a work status of the working vehicle; a sound generation unit that generates a sound signal; a sound output unit that provides the user with sound corresponding to the sound signal generated by the sound generation unit; Further provided with The sound generating unit changes the sound signal depending on the work situation.

7. 2. The work system according to claim 1, The work system further comprises a point awarding unit that awards points to the user in accordance with the work performed by the work vehicle based on the operation of the work vehicle by the user.

8. 2. The work system according to claim 1, a prediction unit that predicts whether the working vehicle will come into contact with an obstacle; a mobile unit control unit that controls the work vehicle so as to avoid contact between the work vehicle and the obstacle when the prediction unit predicts that the work vehicle will come into contact with the obstacle; The working system further comprises:

9. 9. The work system according to claim 8, further comprising an image processing unit capable of performing image processing on the close-up image, the mobile body control unit activates an airbag provided in the working mobile body when contact between the working mobile body and the obstacle cannot be avoided; When the airbag is deployed, the head-mounted display provides the user with the provided image that has been subjected to the image processing, which indicates that the airbag has been deployed.

10. 7. The work system according to claim 6, a data acquisition unit that acquires, via network communication, acoustic data that can be used when generating the acoustic signal; The sound generation unit generates the sound signal using the sound data acquired by the data acquisition unit.

Images