Video transmission system, video transmission device, and video transmission method

Abstract

[Problem] In order to stably transmit video of the periphery of a work machine, even in an environment of variable communication quality, while maintaining operability during remote operation of the work machine, a video transmission system (1) includes: an acquisition unit (11) which acquires video including, in the angle of view, at least a portion of the work machine and at least a portion of a work area of the work machine; a state specification unit (12) which specifies a state of work having, as a target, a work target of the work machine; a region specification unit (13) which specifies a target region in the video in accordance with the specification results; and a transmission unit (14) which transmits adjusted video obtained by adjusting the relative quality between the target region in the video and the outside of the target region.

Description

[Technical field]

[0001] The present invention relates to a technique for transmitting video. [Background technology]

[0002] Patent Document 1 describes a remote control device that displays a work video of a construction machine on a display device at a remote location. The remote control device acquires the gaze position of an operator who operates the machine at a remote location, generates an enlarged image by enlarging a part of the work video related to the gaze position, and displays the enlarged image on the display device.

[0003] Patent Document 2 describes a remote control communication system that captures the entire area around a construction machine with a first camera and captures a specific area in the center of the surrounding area with a second camera. The remote control communication system compresses image data captured by the first camera at a high compression rate and compresses image data captured by the second camera at a low compression rate, and transmits the compressed data to a remote location where an operator is located. At the remote location, the image data is expanded and displayed, thereby displaying the specific area with higher image quality than the surrounding area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-077173 [Patent Document 2] Japanese Patent Application Publication No. 10-112861 Summary of the Invention [Problem to be solved by the invention]

[0005] In the remote control device described in Patent Document 1, in an environment where communication quality fluctuates, there is a possibility that the work video may be distorted or interrupted. Also, in the remote control system described in Patent Document 2, there is a possibility that the area to be focused on during remote control is not included in the specific area, in which case operability during remote control decreases.

[0006] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one example of a purpose of the present invention is to provide technology that stably transmits video of the area around a work machine even in an environment where communication quality fluctuates, while maintaining operability when remotely operating the work machine. [Means for solving the problem]

[0007] A video transmission system according to one aspect of the present invention includes an acquisition means for acquiring video including within an angle of view at least a part of a work machine and at least a part of a work area of ​​the work machine, a status identification means for identifying a status of work performed by the work machine on a work object, an area identification means for identifying a target area in the video in accordance with a result of identification by the status identification means, and a transmission means for transmitting adjusted video obtained by adjusting the relative image quality of the target area and an area outside the target area in the video.

[0008] A video transmission device according to one aspect of the present invention includes an acquisition means for acquiring video including within its angle of view at least a part of a work machine and at least a part of the work area of ​​the work machine, a status identification means for identifying a status of work performed by the work machine on a work object, an area identification means for identifying a target area in the video in accordance with a result of identification by the status identification means, and a transmission means for transmitting adjusted video obtained by adjusting the relative image quality of the target area and an area outside the target area in the video.

[0009] A video transmission method according to one aspect of the present invention includes obtaining video including at least a part of a work machine and at least a part of a work area of ​​the work machine within an angle of view, identifying a work status of the work machine working on a work object, identifying a target area in the video based on the identified work status, and transmitting adjusted video obtained by adjusting the relative image quality of the target area and an area outside the target area in the video. Effect of the Invention

[0010] According to one aspect of the present invention, it is possible to stably transmit video of the surroundings of a work machine even in an environment where communication quality fluctuates, while maintaining operability when remotely operating the work machine. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a configuration of a video transmission system according to a first exemplary embodiment of the present invention. [Diagram 2] 1 is a flow chart showing the flow of a video transmission method according to an exemplary embodiment 1 of the present invention. [Diagram 3] FIG. 2 is a block diagram showing another aspect of the video transmission system according to the first exemplary embodiment of the present invention. [Figure 4] FIG. 2 is a block diagram showing still another aspect of the video transmission system according to the first exemplary embodiment of the present invention. [Diagram 5] FIG. 2 is a block diagram showing still another aspect of the video transmission system according to the first exemplary embodiment of the present invention. [Figure 6] FIG. 11 is a block diagram showing a configuration of a video transmission system according to an exemplary embodiment 2 of the present invention. [Figure 7] 6 is a schematic diagram showing an example of the arrangement of devices constituting the video transmission system shown in FIG. 5. [Figure 8] 6 is a flow chart showing the flow of a video transmission method according to an exemplary embodiment 2 of the present invention. [Figure 9] FIG. 10 is a diagram illustrating an example of an area of ​​interest related to excavation. [Figure 10] FIG. 1 is a diagram illustrating an example of a target area related to excavation. [Figure 11] FIG. 13 is a diagram illustrating an example of a turning-related attention area. [Figure 12] 1 illustrates an example of a region of interest related to turning. [Figure 13] FIG. 10 is a diagram illustrating an example of a traveling-related attention area. [Figure 14] FIG. 2 is a diagram illustrating an example of a target area related to traveling. [Figure 15]FIG. 4 is a diagram illustrating an example of an image displayed by a remote control device. [Figure 16] FIG. 11 is a block diagram showing a configuration of a video transmission system according to an exemplary embodiment 3 of the present invention. [Figure 17] FIG. 11 is a flow chart showing the flow of a video transmission method according to an exemplary embodiment 3 of the present invention. [Figure 18] FIG. 11 is a block diagram showing a configuration of a video transmission system according to each exemplary embodiment 4 of the present invention. [Figure 19] FIG. 11 is a flow chart showing the flow of a video transmission method according to an exemplary embodiment 4 of the present invention. [Figure 20] FIG. 2 is a block diagram showing an example of the physical configuration of devices included in a video transmission system according to each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Example Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described later.

[0013] <Configuration of video transmission system 1> The configuration of a video transmission system 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the video transmission system 1.

[0014] As shown in FIG. 1, the video transmission system 1 includes an acquisition unit 11, a state identification unit 12, an area identification unit 13, and a transmission unit 14. The acquisition unit 11 is an example of a configuration for realizing the acquisition means recited in the claims. The state identification unit 12 is an example of a configuration for realizing the state identification means recited in the claims. The area identification unit 13 is an example of a configuration for realizing the area identification means recited in the claims. The transmission unit 14 is an example of a configuration for realizing the transmission means recited in the claims.

[0015] The acquisition unit 11 acquires an image whose angle of view includes at least a part of the work machine and at least a part of the work area of ​​the work machine. The status identification unit 12 identifies the status of work performed by the work machine on a work target. The area identification unit 13 identifies a target area in the image according to the identification result by the status identification unit 12. The transmission unit 14 transmits an adjusted image obtained by adjusting the relative image quality of the target area and the outside of the target area in the image. The work status, target area, and adjusted image will be explained in detail in the "Flow of video transmission method S1" described later.

[0016] <Flow of video transmission method S1> The flow of the video transmission method S1 executed by the video transmission system 1 configured as above will be described with reference to Fig. 2. Fig. 2 is a flow diagram illustrating the flow of the video transmission method S1. As shown in Fig. 2, the video transmission method S1 includes steps S11 to S14.

[0017] (Step S11) In step S11, the acquisition unit 11 acquires an image including at least a part of the work machine and at least a part of the work area of ​​the work machine in the angle of view. Specifically, the acquisition unit 11 acquires the image captured by the imaging device from the imaging device. The imaging device may be attached to the work machine or may be installed around the work machine. Here, the work area is an area in which the work machine works. The work area may be, for example, an area in which the work machine can operate, an area in which the work machine can move, an area in which a work object may exist, or an area combining some or all of these. In addition, the periphery of the work machine may be any location as long as the work area can be overlooked. For example, the periphery of the work machine may be a ceiling, a pillar, or a beam of the work area if the work machine is indoors. In addition, the periphery of the work machine may be a support erected in the work area, or the eaves or walls of a building if the work machine is outdoors. However, the periphery of the work machine is not limited to these.

[0018] (Step S12) In step S12, the state specification unit 12 specifies the state of the work performed by the work machine on the work object. Here, the state of the work includes, as an example, the type of work. Furthermore, the state of the work includes, as an example, the state of the work part included in the work machine. In this case, the state of the work part includes, as an example, at least one of the position, movement, and the presence or absence of the work object in the work part. Here, the movement of the work part is, for example, a target point sequence (specifically, a set of points or a movement trajectory through which the work part moves when it moves), acceleration, or the direction of speed change, but is not limited to these. Furthermore, the state of the work includes, as an example, the state of other work machines located around the work machine. Furthermore, examples of the work state include the state of the work object, the positional relationship between the work machine and other work machines, the surrounding environment of the work machine, etc. A specific example of the surrounding environment is the size of the area in which the ground is reflected in the image. By referring to the size of such an area, it is possible to determine whether the work state of the work machine is excavating when the work machine is a backhoe. However, the work state is not limited to these.

[0019] Specifically, for example, the condition identifying unit 12 may identify the work condition by at least referring to the analysis result of the video acquired in step S11. Also, for example, the condition identifying unit 12 may identify the work condition by at least referring to information indicating the work phase of the work machine. Here, the information indicating the work phase includes, for example, control information for the work machine based on the operation of the operator (as one example, a steering command).

[0020] (Step S13) In step S13, the region identification unit 13 identifies a target region in the image according to the identification result by the state identification unit 12. Here, the target region is a partial region of the image. Also, the target region is a region to be adjusted in image quality relative to the partial region in the image. For example, if the state of work included in the identification result is a first state, the region identification unit 13 sets the region including the first work part in the image as the target region. Also, if the state of work included in the identification result is a second state, the region identification unit 13 sets the region including the second work part in the image as the target region. Here, the first state and the second state are at least partially different states, and the first work part and the second work part are at least partially different parts. For example, if the work machine is a backhoe, the region including the first work part "bucket" is set as the target region for the first state "digging". Also, for example, the region including the second work part "boom, arm, and bucket" is set as the target region for the second state "turning".

[0021] Here, the area to be focused on during remote operation changes depending on the change in the state of work performed by the work machine. For example, when the work machine turns clockwise, the area to the right, which is the destination of the turn, is desired to be focused on, whereas when the work machine turns counterclockwise, the area to the left is desired to be focused on. For example, the remote operation system described in Patent Document 2 cannot accommodate such changes in the area to be focused on. In contrast, the present exemplary embodiment identifies the target area according to the state of work in this step, and is therefore able to accommodate such changes in the area to be focused on.

[0022] (Step S14) In step S14, the transmitter 14 transmits the adjusted video obtained by adjusting the relative image quality between the target area and the outside of the target area in the video obtained in step S11. For example, the adjusted video is an image obtained by encoding the video obtained in step S11 so that the image quality outside the target area is lower than that of the target area. The image quality may be, for example, but is not limited to, a bit rate, a number of pixels, or a combination of these. Note that the process of generating the adjusted video may be performed by a device included in the video transmission system 1, or may be performed by an external device not included in the video transmission system 1.

[0023] Note that each functional block included in the present exemplary embodiment is not limited to being consolidated in one device, and may be distributed and arranged in a plurality of devices. Also, each functional block included in each exemplary embodiment may be arranged in a cloud. Regardless of the device or cloud in which each of these functional blocks is arranged, it is sufficient that the above-mentioned functions of each functional block can be realized by transmitting and receiving information between these functional blocks as necessary.

[0024] <Advantages of this exemplary embodiment> In this exemplary embodiment, the following configuration is adopted: (i) an image is acquired whose angle of view includes at least a portion of the work machine and at least a portion of the work area of ​​the work machine; (ii) a state of work performed by the work machine on a work object is identified; (iii) a target area is identified in the acquired image based on the identification result; and (iv) an adjusted image obtained by adjusting the relative image quality between the target areas in the image is transmitted.

[0025] According to the above configuration, by transmitting the adjusted image, the amount of communication can be reduced compared to the case of transmitting the acquired image. Therefore, even if the communication quality with the destination decreases, the adjusted image can be transmitted stably. In addition, such a target area can be changed according to the work status. As a result, at the destination, the work machine can be remotely operated while visually recognizing the target area according to the work status with adjusted image quality. Therefore, while maintaining operability when remotely operating the work machine, images of the periphery of the work machine can be transmitted stably even in an environment where communication quality fluctuates.

[0026] <Another aspect 1 of this exemplary embodiment> A video transmission system 1A according to another aspect 1 of the present exemplary embodiment will be described with reference to FIG. 3. FIG. 3 is a block diagram showing a configuration of the video transmission system 1A according to another aspect 1. As shown in FIG. 3, the video transmission system 1A includes a video transmission device 10A and a remote control device 20A. The video transmission device 10A and the remote control device 20A are communicatively connected via a network NW1. The network NW1 may be, for example, a wireless local area network (LAN), a wide area network (WAN), a mobile data communication network (3G, LTE: Long Term Evolution, 4G, 5G, local 5G, etc.), or a combination of these networks. However, the network NW1 is not limited to these.

[0027] Video transmission device 10A includes acquisition unit 11, state identification unit 12, area identification unit 13, and transmission unit 14. Details of each unit are as described above. Video transmission device 10A executes video transmission method S1. Details of video transmission method S1 are as described above. Note that remote control device 20A is applied as the destination of the adjusted video in step S14.

[0028] The remote control device 20A decodes the adjusted image transmitted from the video transmission device 10A and displays it on a display device (not shown). The remote control device 20A also transmits a remote control signal corresponding to operation information input by the operator to the video transmission device 10A. The video transmission device 10A also functions as a control device that controls the operation of the work machine.

[0029] With the above configuration, the video transmission system 1A provides the same effects as those described above.

[0030] <Another aspect 2 of this exemplary embodiment> A video transmission system 1B according to another aspect 2 of the present exemplary embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing a configuration of the video transmission system 1B according to another aspect 2. As shown in Fig. 4, the video transmission system 1B includes a video transmission device 10B and a remote control device 20B. The video transmission device 10B and the remote control device 20B are communicatively connected via a network NW1. A specific example of the network NW1 is as described above.

[0031] Video transmission device 10B includes an acquisition unit 11, an area identification unit 13, and a transmission unit 14. Remote control device 20B includes a state identification unit 12 and a communication unit 220. Details of acquisition unit 11, state identification unit 12, area identification unit 13, and transmission unit 14 are as described above. Communication unit 220 transmits and receives information to and from video transmission device 10B.

[0032] The video transmitting device 10B and the remote control device 20B execute the video transmitting method S1. Details of the video transmitting method S1 are almost as described above, but the following points are different in steps S11 and S12.

[0033] In step S11, the acquisition unit 11 of the video transmission device 10B transmits the acquired video to the remote control device 20B. In step S12, the status identification unit 12 of the remote control device 20B identifies a work status based on the acquired video, and transmits the identified work status to the video transmission device 10B.

[0034] If the communication quality is degraded when transmitting the video from the video transmitting device 10B to the remote control device 20B in step S11, the video decoded by the remote control device 20B may have degraded image quality. However, even if the image quality is not sufficient from the viewpoint of presenting the video for the operator's operation, it may be sufficient for the process of identifying the work status. Therefore, in step S12, the status identification unit 12 arranged in the remote control device 20B can identify the work status.

[0035] In step S12, the status identification unit 12 of the remote control device 20B can also identify the status of the work by referring to the adjusted image transmitted in step S14 of the previous video transmission method S1. This allows the status identification unit 12 to identify the status of the work more accurately than when referring to the low-quality image transmitted in step S11 when communication quality has deteriorated. As a result, the area identification unit 13 of the video transmission device 10B can identify the target area more accurately. In other words, the accuracy of identifying the target area improves with each repetition of the video transmission method S1. Therefore, the operator can gradually and fully grasp the area he or she wants to see, even when low-quality image is initially displayed due to a deterioration in communication quality.

[0036] With the above configuration, the video transmission system 1B can reduce the processing load on the video transmission device 10B arranged on the work machine side, in addition to the effects similar to those described above.

[0037] <Another aspect 3 of this exemplary embodiment> A video transmission system 1C according to another aspect 3 of the present exemplary embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing a configuration of the video transmission system 1C according to another aspect 3. As shown in Fig. 5, the video transmission system 1C includes a video transmission device 10C, a remote control device 20C, and a control device 60C. The remote control device 20C is communicably connected to the video transmission device 10C and the control device 60C via a network NW1.

[0038] The control device 60C is a device that controls the operation of the work machine, and is mounted on the work machine, for example. The control device 60C is a device that inputs operation control signals to each part of the work machine to operate the parts. For example, a case where the work machine operates by autonomous control (a work machine that is not designed to be operated by an operator on board, or a work machine that is designed to operate by receiving an operation control signal from outside the work machine) will be described. In this case, the control device 60C inputs operation control signals corresponding to remote control signals from the remote control device 20C to each part of the work machine. Also, for example, a case where the work machine includes an operation unit (not shown) and operates by attaching an attachment to the operation unit and operating the operation unit will be described. In this case, the control device 60C is connected to the attachment (not shown) so as to be able to communicate with it. The attachment is a device that operates the operation unit in response to a remote control signal from the remote control device 20C. For example, if the operation unit is a rod-shaped operation lever with one end fixed as a fulcrum, the attachment controls the direction and angle of tilt of the operation lever in response to the remote control signal from the remote control device 20C. Furthermore, the control device 60C inputs operation control signals to each section based on the operation of the operation section operated by the attachment.

[0039] The video transmission device 10C is configured in almost the same manner as the video transmission device 10A in the first embodiment. However, unlike the first embodiment, the video transmission device 10C and the control device 60C are configured as separate devices. The video transmission device 10C may be mounted on the work machine or may be disposed in the vicinity of the work machine. The remote control device 20C is configured in almost the same manner as the remote control device 20A in the first embodiment. However, unlike the first embodiment, the remote control device 20C transmits to the control device 60C a remote control signal corresponding to operation information input by the operator.

[0040] The video transmission device 10C and the remote control device 20C execute the video transmission method S1. Details of the video transmission method S1 are as described in the first embodiment.

[0041] Furthermore, the remote control device 20C decodes the adjusted video transmitted in step S14 and displays it on a display device (not shown), and also transmits to the control device 60C a remote control signal corresponding to the operation information input by the operator.

[0042] With the above configuration, the video transmission system 1C can reduce the load on the control device that controls the operation of the work machine in addition to providing the same effects as the first embodiment.

[0043] Other aspects of the present exemplary embodiment Note that this exemplary embodiment is not limited to the above-mentioned aspects, and one or both of the state identification unit 12 and the area identification unit 13 may be arranged in the video transmission device 10 or in the remote control device 20.

[0044] Exemplary embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0045] <Configuration of video transmission system 1D> The configuration of a video transmission system 1D according to this exemplary embodiment will be described with reference to Fig. 6 and Fig. 7. Fig. 6 is a block diagram showing the configuration of the video transmission system 1D. Fig. 7 is a schematic diagram showing an example of the arrangement of devices constituting the video transmission system 1D.

[0046] As shown in FIG. 6, the video transmission system 1D includes a video transmission device 10D, a remote control device 20D, an imaging device 31, a display device 32, and an operation device 33. The operation device 33 is a device for remotely controlling the backhoe 90. For example, when the backhoe 90 operates by autonomous control (when the backhoe 90 is a work machine designed not to be operated by an operator riding on the work machine, or to receive an operation control signal from outside the work machine), the operation device 33 is a device for remotely operating the autonomous control of the backhoe 90. Also, for example, when the backhoe 90 has an operation unit (not shown) and operates by attaching an attachment to the operation unit and operating the operation unit, the operation device 33 is a device for remotely controlling the operation unit. The backhoe 90 will be described in detail later. The video transmission device 10D is connected to the imaging device 31. Also, the remote control device 20D is connected to the display device 32 and the operation device 33. Moreover, the video transmitting device 10D and the remote control device 20D are communicatively connected via a network NW1.

[0047] 7, the video transmission device 10D is mounted on the backhoe 90. In this example, the video transmission device 10D is realized by using the same computer as the control device that controls the operation of the backhoe 90. The remote control device 20D is a device that allows the operator U to remotely control the backhoe 90. The remote control device 20D is installed in a location different from the working area of ​​the backhoe 90.

[0048] (Backhoe 90) A specific configuration of the backhoe 90 will be described. The backhoe 90 excavates soil and sand OBJ and loads it onto the dump truck 80. The backhoe 90 is an example of a "work machine" as set forth in the claims. The soil and sand OBJ is an example of a "work object" as set forth in the claims. The dump truck 80 is an example of an "other work machine" as set forth in the claims. The backhoe 90 operates under the control of the remote control device 20D. As shown in FIG. 7, the backhoe 90 includes a rotating section 91, a movable section 92, and a traveling section 93.

[0049] The traveling unit 93 travels to move the backhoe 90 forward, backward, turn right, and turn left. The traveling unit 93 travels using, for example, an endless track belt.

[0050] The rotating unit 91 is attached to the running unit 93. The rotating unit 91 can rotate about a rotation axis P1 in a plane perpendicular to the paper surface of the figure. When the backhoe 90 is on level ground, the plane perpendicular to the paper surface of Figure 4 is a horizontal plane, and therefore, hereinafter, this plane will be referred to as the "horizontal plane" for convenience.

[0051] The movable unit 92 includes a boom 921 attached to the rotating unit 91, an arm 922 attached to the tip of the boom 921, and a bucket 923 attached to the tip of the arm 922. The boom 921 can reciprocate around a boom axis P2 in a plane approximately perpendicular to the horizontal plane. The arm 922 can reciprocate around an arm axis P3A in the same rotation plane as the boom 921. The bucket 923 can reciprocate around a bucket axis P4 in the same rotation plane as the arm 922. The movable unit 92 is an example of a "working portion" as described in the claims.

[0052] (Imaging device 31) In the example shown in FIG. 7, the imaging device 31 is mounted on a backhoe 90. The imaging device 31 is installed so that (i) at least a part of the movable part 92 (boom 921, arm 922, and bucket 923) and the rotating part 91, and (ii) at least a part of the work area including the soil OBJ are included in the angle of view. As a result, the imaging device 31 generates an image including these as subjects. The image generated by the imaging device 31 is a moving image. In other words, the image includes a time series of a plurality of still images. Note that the imaging direction of the imaging device 31 may be fixed or may be changeable. In that case, for example, the imaging direction of the imaging device 31 may be changed by a remote control signal from the remote control device 20D. In this case, it is desirable that the imaging direction of the imaging device 31 is changed within a range that includes at least the movable part 92, the rotating part 91, and the soil OBJ in the angle of view. Although one imaging device 31 is shown in FIG. 7, the video transmission device 10D may obtain videos from a plurality of imaging devices 31, respectively.

[0053] (Video transmitting device 10D) 6, the video transmission device 10D includes a control unit 110D and a communication unit 120D. The communication unit 120D transmits and receives information to and from a remote control device 20D via a network NW1. The control unit 110D includes an acquisition unit 11D, a state identification unit 12D, a region identification unit 13D, a transmission unit 14D, and an adjustment unit 15D.

[0054] In addition to these functional blocks 11D to 15D, the control unit 110D has a function of controlling each part of the backhoe 90 according to a remote control signal received from the remote control device 20D. For example, a case where the backhoe 90 operates by autonomous control will be described. In this case, the control unit 110D inputs an operation control signal corresponding to the remote control signal received from the remote control device 20D to each part of the backhoe 90. Also, for example, a case where the backhoe 90 has an operation unit for an operator to operate will be described. In this case, the control unit 110D inputs an operation control signal corresponding to the operation of the operation unit (not shown) operated by the attachment (not shown) according to the remote control signal to each part of the backhoe 90. For example, the control unit 110D rotates a part or all of the rotating unit 91, the boom 921, the arm 922, and the bucket 923 according to the remote control signal, thereby causing the backhoe 90 to perform work such as excavation, rotation, and dumping. Also, for example, the control unit 110D causes the traveling unit 93 to travel, thereby causing the backhoe 90 to travel, in accordance with a remote control signal.

[0055] The adjustment unit 15D generates an adjusted image in which the image quality outside the target area is adjusted to be relatively low with respect to the target area in the image acquired from the imaging device 31. The state identification unit 12D, the acquisition unit 11D, and the transmission unit 14D are configured in substantially the same manner as the functional blocks of the same names in the exemplary embodiment 1. Details of the operation of each functional block included in the control unit 110D will be described in "Flow of video transmission method S1D" to be described later.

[0056] (Remote control device 20D) The remote control device 20D includes a control unit 210D and a communication unit 220D. The communication unit 220D transmits and receives information to and from the video transmission device 10D via the network NW1. The control unit 210D includes a decoding unit 21D, a display unit 22D, and a reception unit 23D.

[0057] The decoding unit 21D decodes the adjusted video received from the video transmission device 10D. The display unit 22D displays the video decoded by the decoding unit 21D on the display device 32. The reception unit 23D receives operation information input by the operator U using the operation device 33. The reception unit 23D also transmits a remote control signal corresponding to the operation information to the video transmission device 10D. Here, the operation information includes commands for each part indicated by the inclination, acceleration, direction, etc. of the operation lever if the operation device 33 is a rod-shaped operation lever with one end fixed, for example.

[0058] <Flow of video transmission method S1D> The flow of a video transmission method S1D executed by the video transmission system 1D configured as above will be described with reference to Fig. 8. Fig. 8 is a flow diagram illustrating the flow of the video transmission method S1D. As shown in Fig. 8, the video transmission method S1D includes steps S11D to S19D.

[0059] (Step S11D) In step S11D, the acquisition unit 11D acquires the video captured by the imaging device 31.

[0060] (Step S12D) In step S12D, the state specifying unit 12D specifies the state of the work performed by the backhoe 90 on the soil OBJ. Here, the state of the work includes the state of the bucket 923 and the state of the dump truck 80. The state of the bucket 923 includes the position and movement of the bucket 923, and the presence or absence of soil OBJ in the bucket 923. In other words, the state specifying unit 12D specifies the state of the work according to one or both of the state of the bucket 923 and the state of the dump truck 80. Examples of such work states include, but are not limited to, "during digging," "just before turning," "during turning," "just before dumping," "during dumping," "just before traveling," and "traveling."

[0061] (Example of a process for identifying the status of an operation) A specific example of the process for identifying the work status will be described. For example, the status identifying unit 12D identifies the work status by grasping the work currently being performed by the backhoe 90. In addition, for example, the status identifying unit 12D may identify the work status by predicting the work that the backhoe 90 will perform in the future.

[0062] For example, if the work currently being performed by the backhoe 90 is "digging", the state specifying unit 12D specifies the work state as "digging". Also, if the work predicted to be performed next by the backhoe 90 is "turning", the state specifying unit 12D specifies the work state as "just before turning".

[0063] For example, "during excavation" refers to a series of actions from inserting the bucket 923 into the excavation point, tilting the bucket 923 to load soil OBJ into the bucket 923, and lifting the bucket 923. Also, "just before turning" refers to the time immediately before the turning part 91 turns.

[0064] For example, the state identification unit 12D refers to the analysis results of the image acquired in step S11D and determines condition (i) "whether the image includes a bucket 923, and whether there is soil and sand OBJ in the bucket 923" and condition (ii) "whether the image includes the bucket 923 and the dump truck 80, and whether the distance between the bucket 923 and the dump truck 80 is equal to or greater than a threshold value."

[0065] (Example of drilling in progress) The status identification unit 12D may identify the status of the work as “digging in progress” if condition (i) is not satisfied (there is no soil OBJ in the bucket 923) and condition (ii) is satisfied (the bucket 923 and the dump truck 80 are separated).

[0066] (Example immediately before turning) The state identification unit 12D may identify the work state as “just before turning” when condition (i) is satisfied (soil and sand OBJ are present in the bucket 923) and condition (ii) is satisfied (the bucket 923 and the dump truck 80 are separated).

[0067] (Example during turning) The state specifying unit 12D may specify that the work state is "turning" when the condition (i) is not satisfied (there is no soil or sand OBJ in the bucket 923) and the condition (ii) is not satisfied (the bucket 923 and the dump truck 80 are close). Note that "turning" means that the turning unit 91 is turning.

[0068] (Examples of soil release) When condition (i) is satisfied (soil OBJ is present in the bucket 923) and condition (ii) is not satisfied (the bucket 923 and the dump truck 80 are close), the status identification unit 12D may identify the work status as “just before releasing soil” or “in the process of releasing soil.”

[0069] (Example while driving) Furthermore, the state identification unit 12D may identify that the backhoe 90 is "traveling" based on a change in the image or a change in the position information of the backhoe 90, without being limited to the above conditions (i) and (ii). An example of a change in the image is a change in the position information of the recognized soil / sand OBJ in the image.

[0070] The conditions for identifying the state of work are not limited to the above-mentioned "specific example of excavation," "specific example immediately before turning," "specific example of turning," "specific example of dumping soil," and "specific example of traveling." The conditions for identifying the state of work may be identified using other conditions.

[0071] (Step S13D) In step S13D, the region identifying section 13D identifies a target region in the video by referring to the identification result by the state identifying section 12D.

[0072] As a specific example of the process, the area specifying unit 13D specifies the positions of the boom 921, the arm 922, and the bucket 923 in the image. The area specifying unit 13D also specifies a target area according to the state of the work by referring to the specified positions of each part. A specific example of the target area will be described.

[0073] (Specific examples of areas of interest related to drilling) A specific example of a target area when performing work related to excavation using a backhoe 90 will be described with reference to Figs.

[0074] FIG. 9 is a diagram for explaining an example of an attention area of ​​an operator U when performing work related to excavation using a backhoe 90, and is a top view of the backhoe 90. As shown in FIG. 9, when performing work related to excavation, the operator U pays attention to an attention area R1 around the bucket 923 including the bucket 923. When the state of the work related to excavation is specified, the area specification unit 13D specifies a target area so as to correspond to such attention area R1. Here, the attention area R1 is an area that the operator U pays attention to depending on the state of the work. In contrast, the target areas R1a to R1c described later are areas in the image whose image quality is to be adjusted. Note that the attention area R1 is illustrated for the purpose of explaining the target area, and does not necessarily need to be specified by the area specification unit 13D. In other words, unlike the device described in Patent Document 1, the area specification unit 13D does not need to specify the line of sight of the operator U. The area specification unit 13D specifies the target areas R1a to R1c described below in the image.

[0075] FIG. 10 is a diagram for explaining an example of a target area when the state of work related to excavation is specified. For example, the state of work related to excavation includes "just before excavation (state before the teeth of the bucket are inserted into the soil)", "excavating", or "excavation completed (state in which the bucket 923 carrying the soil OBJ is lifted up to a predetermined height)". For example, when "just before excavation" is specified as the state of work, the area specifying unit 13D specifies a target area R1a in the image G1a. The target area R1a is an area that includes the soil OBJ directly below the bucket 923 as a subject. By clearly displaying such a target area R1a, the operator U can confirm the shape of the soil OBJ, determine the position to lower the bucket 923, and operate the operation device 33.

[0076] Furthermore, when "digging" is identified as the work status, the region identification unit 13D identifies a target region R1b in the image G1b. The target region R1b is a region that includes the boom 921, the arm 922, the bucket 923, and the soil OBJ in front of the bucket 923 as subjects. By clearly displaying such target region R1b, the operator U can confirm the shape of the soil OBJ in front of the bucket 923 and the positional relationship of the joints, and can operate the operation device 33 by determining the operation of the bucket 923.

[0077] Furthermore, when "excavation completed" is identified as the work status, the region identification unit 13D identifies a target region R1c in the image G1c. The target region R1c is a region that includes the bucket 923 and the soil OBJ in the bucket 923 as subjects. With such a target region R1c clearly displayed, the operator U can operate the operation device 33 to adjust the amount of soil OBJ in the bucket 923 or to move the bucket 923 while checking whether the soil OBJ put in the bucket 923 is about to spill.

[0078] (Specific examples of areas of interest and areas of interest related to turning) Specific examples of the attention area and the target area when performing work related to turning with the backhoe 90 will be described with reference to Figs. 11 and 12.

[0079] FIG. 11 is a top view of the backhoe 90, which is a diagram for explaining an example of an attention area of ​​the operator U when performing work related to turning using the backhoe 90. As shown in FIG. 10, when performing work related to turning, the operator U focuses on an attention area R2 that includes the bucket 923 and extends to the left from the bucket 923. Here, it is assumed that the bucket 923 turns in a counterclockwise direction. When the state of the work related to turning is specified, the area specifying unit 13D specifies a target area so as to correspond to such attention area R2. Note that the attention area R2 is a spatial area illustrated for the purpose of explaining the target area, and does not necessarily need to be specified by the area specifying unit 13D. The area specifying unit 13D specifies target areas R2a to R2b, which will be described below, in the image.

[0080] FIG. 12 is a diagram for explaining an example of a target area when a work state related to turning is identified. For example, the work state identified in relation to turning includes "just before turning" or "during turning". For example, when "just before turning" is identified as the work state, the area identification unit 13D identifies a target area R2a in the image G2a. The target area R2a is an area including the boom 921, the arm 922, the bucket 923, and the soil OBJ as objects. By clearly displaying such a target area R2a, the operator U can check the situation around the boom 921, determine whether it is safe, and operate the operation device 33 to start turning.

[0081] Furthermore, when "turning" is identified as the work state, the region identification unit 13D identifies a target region R2b in the image G2b. The target region R2b includes the boom 921, the arm 922, and the bucket 923 as objects, and is a region extending to the left from each of these parts. By clearly displaying such target region R2b, the operator U can operate the operation device 33 to continue turning while judging whether the turning direction (counterclockwise direction) is safe.

[0082] (Specific examples of driving-related areas of interest and target areas) Specific examples of the attention area and the target area when performing a task related to driving will be described with reference to Figs. 13 and 14.

[0083] FIG. 13 is a top view of the backhoe 90, which is a diagram for explaining an example of an attention area of ​​the operator U when performing work related to traveling using the backhoe 90. As shown in FIG. 13, when performing work related to traveling, the operator U focuses on an attention area R3 including the traveling direction of the backhoe 90. When the state of the work related to traveling is identified, the area identification unit 13D identifies a target area so as to correspond to such attention area R3. Note that the attention area R3 is a spatial area illustrated for the purpose of explaining the target area, and does not necessarily need to be identified by the area identification unit 13D. The area identification unit 13D identifies a target area R3a, which will be described below, in the image.

[0084] FIG. 14 is a diagram illustrating an example of a target area when a task related to traveling is identified. For example, the state of the task related to traveling includes "just before traveling forward." For example, when "just before traveling forward" is identified as the task state, the area identification unit 13D identifies a target area R3a in the image G3a. The target area R3a is an area including the boom 921, the arm 922, and the bucket 923 as objects. By clearly displaying such a target area R3a, the operator U can operate the operation device 33 to start traveling while judging whether the front is safe.

[0085] (Step S14D) In step S14D, the adjustment unit 15D generates an adjusted image in which the image quality outside the target area is adjusted in the image acquired in step S11D so that the image quality is relatively low compared to the target area. Specific examples of the image quality to be adjusted are as described in the exemplary embodiment 1. Note that a known technology can be adopted as a technology for adjusting and encoding the relative image quality between the target area and the area outside the target area.

[0086] (Step S15D) The transmitter 14D transmits the generated adjusted image to the remote control device 20D. Such adjusted image requires less communication traffic than the image before adjustment. Therefore, even if the communication quality fluctuates in this step, the adjusted image is more likely to be transmitted satisfactorily.

[0087] (Step S16D) In step S16D, the decoding section 21D of the remote control device 20D decodes the received adjusted video.

[0088] (Step S17D) In step S17D, display unit 22D displays the decoded image. A specific example of the displayed image will now be described with reference to FIG. 15. FIG. 15 is a diagram showing a specific example of an image displayed on display device 32 by remote control device 20D. In FIG. 15, image G4 displayed on display device 32 includes target area R4. Image G4 is a decoded image after adjustment in which the image quality outside target area R4 is adjusted to be relatively low compared to target area R4. Therefore, the image quality outside target area R4 is relatively low compared to target area R4. On the other hand, the image quality of target area R4 can be expected to be clear.

[0089] (Step S18D) In step S18D, the reception unit 23D receives operation information input by the operator U using the operation device 33. The operator can accurately perform an operation according to the state of the work because the target area R4 is clearly displayed. The reception unit 23D also transmits a remote control signal corresponding to the operation information to the video transmission device 10D.

[0090] (Step S19D) In step S19D, control unit 110D of video transmission device 10D controls the operation of backhoe 90 in accordance with the received remote control signal.

[0091] <Advantages of this exemplary embodiment> In addition to the same effects as those of the first exemplary embodiment, this exemplary embodiment also has the following effects.

[0092] In this exemplary embodiment, a configuration is adopted in which the work state identified by the state identifying unit 12D includes the state of the bucket 923 and the like provided on the backhoe 90. With this configuration, the state identifying unit 12D can more accurately determine the work state including the state of the bucket 923, such as "digging," "just before turning," "turning," and the like. As a result, the target area can be more accurately determined according to the state of the bucket 923 and the like.

[0093] In addition, in this exemplary embodiment, a configuration is adopted in which the work state identified by the state identifying unit 12D includes the state of the dump truck 80, which is another work machine. With this configuration, the state identifying unit 12D can more accurately determine the work state including the state of the dump truck 80, such as "just before dumping soil," "during dumping soil," etc. As a result, the target area according to the state of the dump truck 80 can be more accurately determined.

[0094] In addition, in this exemplary embodiment, the state identification unit 12D is configured to identify the state of the work by at least referring to the analysis result of the video. With this configuration, the state identification unit 12D can more accurately determine the state of the work based on the information obtained from the video. As a result, the target area according to the state of the work can be more accurately determined.

[0095] Exemplary embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first and second exemplary embodiments are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0096] <Configuration of video transmission system 1E> The configuration of a video transmission system 1E according to this exemplary embodiment will be described with reference to Fig. 16. Fig. 16 is a block diagram showing the configuration of the video transmission system 1E.

[0097] 16, the video transmission system 1E is configured in a manner similar to that of the video transmission system 1D according to the exemplary embodiment 2. However, the video transmission system 1E is different in that the video transmission system 1E includes a video transmission device 10E instead of the video transmission device 10D, and a remote control device 20E instead of the remote control device 20D. The other points are the same as those described in the exemplary embodiment 2.

[0098] Here, in the second exemplary embodiment, the state identification unit 12D and the area identification unit 13D are arranged in the video transmission device 10D, whereas in this exemplary embodiment, the state identification unit 24E and the area identification unit 25E are arranged in the remote control device 20E.

[0099] The video transmission device 10E includes an acquisition unit 11E, a transmission unit 14E, and an adjustment unit 15E. The acquisition unit 11E, the transmission unit 14E, and the adjustment unit 15E are configured in substantially the same manner as the functional blocks of the same names in the second exemplary embodiment.

[0100] The remote control device 20E includes a decoding unit 21E, a display unit 22E, a reception unit 23E, a state identification unit 24E, a region identification unit 25E, an object recognition unit 26E, and a vector determination unit 27E. The decoding unit 21E, the display unit 22E, and the reception unit 23E are configured in substantially the same manner as the functional blocks of the same names in the exemplary embodiment 2. The state identification unit 24E is configured in substantially the same manner as the state identification unit 12D in the exemplary embodiment 2, but differs in that it refers to the recognition result by the object recognition unit 26E. The region identification unit 25E is configured in substantially the same manner as the region identification unit 13D in the exemplary embodiment 2, but differs in that it further refers to the determination result by the vector determination unit 27E. Details of the state identification unit 24E, the region identification unit 25E, the object recognition unit 26E, and the vector determination unit 27E will be described in "Flow of the video transmission method S1E" to be described later.

[0101] <Video transmission method S1E flow> The flow of a video transmission method S1E executed by the video transmission system 1E configured as above will be described with reference to Fig. 17. Fig. 17 is a flow diagram illustrating the flow of the video transmission method S1E. As shown in Fig. 17, the video transmission method S1E includes steps S11E to S19E. The processing of steps S14E to S19E is similar to steps S14D to S19D in the video transmission method S1D according to the exemplary embodiment 2. Here, steps S11E, S12-1E, S12-2E, S12-3E, and S13E will be described.

[0102] (Step S11E) In step S11E, the acquisition unit 11E acquires the video captured by the imaging device 31. The acquisition unit 11E also transmits the acquired video to the remote control device 20E.

[0103] (Step S12-1E) In step S12-1E, the object recognition unit 26E of the remote control device 20E refers to the image acquired in step S11E and recognizes objects contained in the image. The object recognition unit 26E is a functional block for recognizing objects contained in the image. The object recognition unit 26E can employ a known technology for recognizing objects contained in the image. For example, the object recognition unit 26E recognizes objects contained in the image from among the working parts (boom 921, arm 922, bucket 923, etc.) of the backhoe 90, the dump truck 80, the soil OBJ, etc.

[0104] (Step S12-2E) In step S12-2E, state specifying unit 24E refers to the recognition result by object recognition unit 26E (ie, the analysis result of the image) and specifies the state of the work performed by backhoe 90 on earth and sand OBJ.

[0105] For example, when the recognition result by the object recognition unit 26E indicates the bucket 923 and the soil OBJ in the bucket 923, the state identification unit 24E may identify the work state as "immediately before releasing soil", which is about to start releasing soil.

[0106] (Step S12-3E) In step S12-3E, vector determination unit 27E refers to the video acquired in step S11E and determines the direction in which the area in which the same subject is captured moves (hereinafter, referred to as vector). The video acquired in step S11E may include a time series of multiple still images. Thus, vector determination unit 27E extracts an area in which the same subject is captured in the multiple still images and determines the vector of the area. Vector determination unit 27E may determine the vector using a time series of as many still images as necessary to determine the vector. Note that the execution order of steps S12-2E and S12-3E does not necessarily have to be in this order.

[0107] (Step S13E) In step S13E, region identifying unit 25E refers to the identification result (task state) by state identifying unit 24E and the determination result (vector) by vector determining unit 27E to identify a target region in the video acquired in step S11E. Region identifying unit 25E also transmits the identification result to video transmitting device 10E.

[0108] As an example, the region identification unit 25E may expand the target region according to the state of work toward the vector of the determination result. For example, when the state of work is "digging" and the vector is "upward" in the image, the target region may be an area obtained by expanding the target region R1b in the image G1b shown in FIG. 10 toward the upward direction. The region identification unit 25E may also move the target region according to the state of work based on the vector of the determination result. As a result, the target region moves, for example, following the movement of the bucket 923.

[0109] <Advantages of this exemplary embodiment> In addition to the same effects as those of the first exemplary embodiment, this exemplary embodiment has the following effects.

[0110] In this exemplary embodiment, the state identification unit 24E is configured to refer to the recognition result (i.e., the analysis result of the image) by the object recognition unit 26E and identify the state of the work on the soil OBJ by the backhoe 90. This configuration makes it possible to more accurately determine the state of the work based on the object recognized from the image.

[0111] In addition, in this exemplary embodiment, the area identifying unit 25E identifies the target area in the video acquired in step S11E by referring to the identification result (task state) by the state identifying unit 24E and the determination result (vector) by the vector determining unit 27E. With this configuration, the target area can be determined with higher accuracy because the movement in the video is taken into consideration in addition to the task state.

[0112] In addition, in this exemplary embodiment, the condition specifying unit 24E is disposed in the remote control device 20E. This reduces the load on the video transmission device 10E compared to the case where the condition specifying unit 24E is disposed in the video transmission device 10E.

[0113] Exemplary embodiment 4 A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to third exemplary embodiments are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0114] <Configuration of video transmission system 1F> The configuration of a video transmission system 1F according to this exemplary embodiment will be described with reference to Fig. 18. Fig. 18 is a block diagram showing the configuration of the video transmission system 1F.

[0115] 18, the video transmission system 1F is configured in a manner similar to that of the video transmission system 1E according to the exemplary embodiment 3. However, the difference is that the video transmission system 1F includes a video transmission device 10F instead of the video transmission device 10E, and includes a remote control device 20F instead of the remote control device 20E. The other points are the same as those described in the exemplary embodiment 3.

[0116] The video transmitting device 10F includes an acquiring unit 11F, a transmitting unit 14F, and an adjusting unit 15F. The acquiring unit 11F, the transmitting unit 14F, and the adjusting unit 15F are configured in substantially the same manner as the functional blocks of the same names in the third exemplary embodiment.

[0117] The remote control device 20F includes a decoding unit 21F, a display unit 22F, a reception unit 23F, a state identification unit 24F, a region identification unit 25F, and a prediction unit 28F. The prediction unit 28F is an example of a configuration that realizes the prediction means described in the claims. The decoding unit 21F, the display unit 22F, the reception unit 23F, the state identification unit 24F, and the region identification unit 25F are configured almost similarly to the functional blocks of the same names in the exemplary embodiment 3. However, the state identification unit 24F differs in that it at least refers to information indicating the work phase by the backhoe 90. Also, the region identification unit 25F differs in that it further refers to the prediction result by the prediction unit 28F. Details of the state identification unit 24F, the region identification unit 25F, and the prediction unit 28F will be described in "Flow of the video transmission method S1F" described later.

[0118] <Video transmission method S1F flow> The flow of a video transmission method S1F executed by the video transmission system 1F configured as above will be described with reference to Fig. 19. Fig. 19 is a flow diagram illustrating the flow of the video transmission method S1F. As shown in Fig. 19, the video transmission method S1F includes steps S11F to S19F. The processes of steps S11F, S14F to S19F are similar to steps S11E, S14E to S19E in the video transmission method S1E according to the exemplary embodiment 3. Here, steps S12-1F, S12-2F, S12-3F, and S13F will be described.

[0119] (Step S12-1F) In step S12-1F, the state identification unit 24F acquires a history of operation information. The operation information is information input by the operator U to the operation device 33 to remotely control the backhoe 90. The operation information includes, for example, information on (i) opening and closing the bucket 923, (ii) folding / extending the first joint of the boom 921, (iii) folding / extending the second joint of the boom 921, (iv) turning, or (v) traveling. However, the operation information is not limited to this. The history of operation information acquired here is information that has been received by the reception unit 23F up to now.

[0120] (Step S12-2F) In step S12-2F, the state identification unit 24F identifies the state of work by at least referring to the information indicating the work phase. Specifically, the information indicating the work phase includes information indicating the timing at which the work phase switches. For example, the state identification unit 24F identifies the timing at which the work phase switches based on the history of the operation information. The state identification unit 24F identifies the state of work by the backhoe 90 based on the timing at which the work phase switches. For example, the work phases are "digging," "turning," "discharging," and "traveling," and it is assumed that the work is performed in this order.

[0121] (Example 1: Digging and turning) As a specific example, if, for example, in the operation information history, the above-mentioned operation information (i), (ii), and (iii) appear with a frequency equal to or greater than a threshold, and then operation information (iv) appears, the state identification unit 24F identifies this as timing t1 when the work switches from excavation to turning.

[0122] (Example 2: Rotation and release) Furthermore, in the operation information history, if operation information (i), (ii), and (iii) appear with a frequency equal to or greater than a threshold value after operation information (iv) described above in specific example 1 appears, the state identification unit 24F identifies this as timing t2 when the operation switches from turning to dumping.

[0123] (Example 3: Turning from soil release) Furthermore, in the operation information history, if the operation information (i), (ii), and (iii) described above in specific example 2 appears with a frequency equal to or greater than a threshold, and then operation information (iv) appears, the state identification unit 24F identifies this as timing t3, when the operation switches from dumping to turning.

[0124] (Example 4: Turning and digging) Furthermore, in the operation information history, if operation information (i), (ii), and (iii) appear with a frequency equal to or greater than a threshold value after operation information (iv) described above in specific example 3 appears, the state identification unit 24F identifies this as timing t4 when the operation switches from turning to excavation.

[0125] (Example 5: Driving) Furthermore, when the operation information of (v) described above appears following Specific Example 4 in the operation information history, the state identifying unit 24F identifies this as timing t5 when the work switches to traveling.

[0126] In this way, the state identification unit 24F identifies the timing at which the work phase switches based on the history of the operation information, and identifies the work state by referring to the identified timing. For example, the state identification unit 24F can identify that the work state is "turning" between the above-mentioned timings t1 and t2. The state identification unit 24F also transmits the identification result to the video transmission device 10F.

[0127] (Step S12-3F) In step S12-3F, the prediction unit 28F predicts the operation of the backhoe 90. For example, the prediction unit 28F refers to the analysis result of the video and acquires the direction in which the bucket 923 moved in the video. Furthermore, the prediction unit 28F predicts the direction in which the bucket 923 will move from now (predicted direction) based on the acquired moving direction. Furthermore, for example, the prediction unit 28F may predict the operation of the backhoe 90 by referring to information linking the history of operation information with the amount of movement of a part of the backhoe 90 (for example, the bucket 923) corresponding to the history. Furthermore, for example, the prediction unit 28F may predict the operation of the backhoe 90 by identifying the speed and width at which a part of the backhoe 90 (for example, the bucket 923) moves on the screen on which the video is displayed.

[0128] (Step S13F) In step S13F, the region identifying unit 25F identifies a target region in the image acquired in step S11F by referring to the identification result (state of work) by the state identifying unit 24F and the prediction result (prediction direction) by the prediction unit 28F. As an example, the region identifying unit 25F may expand the target region according to the state of work in the predicted direction. For example, when the state of work is "digging" and the predicted direction is "upward" in the image, the target region may be an area obtained by expanding the target region R1b in the image G1b shown in FIG. 10 by the region size of the bucket 923 in the upward direction. The region identifying unit 25F may also move the target region according to the state of work based on the predicted direction. As a result, the target region moves, for example, following the predicted movement of the bucket 923.

[0129] In this exemplary embodiment, the state identification unit 24F may identify the state of the work by referring to a database that aggregates the history of operation information by multiple operators, not limited to the history of operation information by one operator U. The multiple operators may be, for example, operators who remotely operate one backhoe 90 at different times, or operators who remotely operate multiple backhoes 90. The state identification unit 24F may also identify the state of the work by using an estimation model that inputs the history of operation information and outputs the state of the work. Such an estimation model may also be generated by machine learning using the history of operation information by such multiple operators U as teacher data.

[0130] <Advantages of this exemplary embodiment> In addition to the same effects as those of the first exemplary embodiment, this exemplary embodiment has the following effects.

[0131] In this exemplary embodiment, the state identification unit 24F is configured to identify the state of work by at least referring to information indicating the work phase of the backhoe 90. For example, when it is assumed that a plurality of work phases are performed in sequence, the state identification unit 24F can accurately identify the state of work by identifying the timing at which the work phases change.

[0132] In addition, in this exemplary embodiment, the state identification unit 24F determines the change of the work phase based on the history of the operation information by the operator U, and identifies the work state based on the determination result. Compared to the case of performing a process that refers to the analysis result of the video, the process of referring to the history of the operation information often has a lighter load. Therefore, the load of the process of identifying the work state can be reduced.

[0133] In this exemplary embodiment, the prediction unit 28F predicts the movement of the backhoe 90 (bucket 923), and the area identification unit 25F identifies the target area by referring to the identification result (work state) by the state identification unit 24F and the prediction result by the prediction unit 28F. With this configuration, the predicted movement of the backhoe 90 is taken into consideration in addition to the work state, so that the target area can be determined with higher accuracy so as to match the area that the operator U will be paying attention to according to the work that the backhoe 90 will be performing.

[0134] [Variation 1] Each of the exemplary embodiments described above can be modified as follows. In each of the exemplary embodiments, the acquisition unit 11 (11D, 11E, 11F) may further acquire communication status information indicating the communication status in the transmission path used by the transmission unit 14 (14D, 14E, 14F). In this case, the transmission unit 14 (14D, 14E, 14F) refers to the acquired communication status information to specify the relative image quality between the target area and the outside of the target area. For example, the transmission unit 14 (14D, 14E, 14F) may specify that there is no difference in image quality between the target area and the outside of the target area as the relative image quality. In this case, the transmission unit 14 (14D, 14E, 14F) transmits the video acquired by the acquisition unit 11 (11D, 11E, 11F) as it is. Also, for example, the transmission unit 14 (14D, 14E, 14F) may specify that the image quality of the target area is higher than that of the outside of the target area as the relative image quality. The relative image quality is represented, for example, by adjustment parameters for the target area of ​​the image and its outside. In this case, the transmitter 14 (14D, 14E, 14F) transmits the adjusted image. In other words, the transmitter 14 (14D, 14E, 14F) may switch between transmitting the image acquired by the acquisition unit 11 (11D, 11E, 11F) as is or transmitting the adjusted image by referring to the communication status information.

[0135] In this case, the communication status information may include information predicting a future communication status, and may also include a history of the communication status up to now.

[0136] By modifying in this way, each exemplary embodiment can clearly present the inside of the target area using the adjusted image when the communication situation is below a predetermined quality, while clearly presenting the entire unadjusted image to the operator U when the communication situation is above the predetermined quality. As a result, the operability of the operator is further improved.

[0137] [Modification 2] Furthermore, in each of the exemplary embodiments described above, the region identification unit 13 (13D, 13E, 13F) may vary the size of the target region depending on the communication quality. For example, the region identification unit 13 (13D, 13E, 13F) may reduce the size of the target region as the communication quality deteriorates. Furthermore, the region identification unit 13 (13D, 13E, 13F) may vary the image quality inside and outside the target region depending on the communication quality. For example, the region identification unit 13 (13D, 13E, 13F) may lower the image quality outside the target region as the communication quality deteriorates.

[0138] By modifying in this way, each of the exemplary embodiments can more stably transmit an adjusted image for clearly displaying at least the target area.

[0139] [Modification 3] In the fourth exemplary embodiment, the reception unit 23F may acquire operation information including the voice of the operator U. In this case, the state identification unit 24F identifies the state of the work by the backhoe 90 by referring to the history of the operation information including the voice.

[0140] [Modification 4] The above-mentioned exemplary embodiments 2 to 4 have been described mainly with reference to examples in which the backhoe 90 is applied to the work machine. However, the exemplary embodiments can achieve the same effects even if a device other than the backhoe 90 is applied to the work machine. For example, other construction machines may be applied to the work machine. Specific examples of other construction machines include cranes, forklifts, rollers, bulldozers, etc. However, specific examples of construction machines are not limited to these. Furthermore, an automatic guided vehicle (AGV) may be applied to the work machine.

[0141] Moreover, in the exemplary embodiments 2 to 4, the operator U does not necessarily have to be a person. For example, the operator U may be a computer, a robot, or the like having an AI (Artificial Intelligence) function that operates a work machine, but is not limited thereto. Furthermore, in the exemplary embodiments 2 to 4, the work object is not limited to soil OBJ. For example, it may be soil, sand, dirt, snow, grain, cement, or the like, or may be a granular or amorphous solid or liquid. For example, if the work machine is a crane, the work object is a load slinging on the crane.

[0142] [Modification 5] The above-mentioned exemplary embodiments 2 to 4 have been described mainly with reference to examples in which the dump truck 80 is applied to other work machines. However, other backhoes, rollers, cranes, etc. may be applied to the other work machines.

[0143] [Modification 6] Moreover, the above-described exemplary embodiments 2 to 4 may include the video transmission devices 10D, 10E, 10F and the remote control devices 20D, 20E, and 20F, each of which is tied to a plurality of backhoes 90. In this case, each of the remote control devices 20D, 20E, and 20F transmits a remote control signal to the backhoe 90 tied to the device itself. Also, each of the video transmission devices 10D, 10E, and 10F transmits a video related to the backhoe 90 tied to the device itself to the corresponding remote control device 20D, 20E, and 20F. In this case, the video transmission devices 10D, 10E, and 10F tied to some of the backhoes 90 among the plurality of backhoes 90 may transmit the adjusted video. Also, the other video transmission devices 10D, 10E, and 10F may transmit the video as it is without generating the adjusted video. For example, a server (not shown) may determine the backhoe 90 among the plurality of backhoes 90 to transmit the adjusted video. Such a determination may be made based on the importance of the work status of each backhoe 90. By modifying in this way, when multiple backhoes 90 are working in the same work area, the image of the backhoe 90 with the higher importance among the multiple backhoes 90 can be transmitted with priority while maintaining operability.

[0144] [Modification 7] In the above-mentioned exemplary embodiment 4, the state identification unit 24F may further refer to the analysis result of the video to identify the state of the work, if necessary. Here, the process of referring to the history of the operation information is considered to have a lighter load than the video analysis process. For example, when the state identification unit 24F cannot identify the state of the work by referring to the history of the operation information, the state identification unit 24F may identify the state of the work by referring to the analysis result of the video. The process of identifying the state of the work by referring to the analysis result of the video is as described in the exemplary embodiments 2 and 3. By modifying in this way, it is possible to improve the reliability of the identified state of the work while reducing the load of the process of identifying the state of the work.

[0145] [Variation 8] In the above-described exemplary embodiments 2 to 4, each functional block may be disposed in the video transmission device 10D, 10E, or 10F, or in the remote control device 20D, 20E, or 20F.

[0146] For example, in the second exemplary embodiment, one or both of the state identification unit 12D and the region identification unit 13D may be disposed in the video transmission device 10D or in the remote control device 20D. However, it is preferable that the region identification unit 13D can refer to the identification result by the state identification unit 12D without going through the network NW1. Therefore, it is preferable that both the state identification unit 12D and the region identification unit 13D are disposed in the video transmission device 10D or in the remote control device 20D.

[0147] Also, for example, in the exemplary embodiment 3, some or all of the state identification unit 24E, the area identification unit 25E, the object recognition unit 26E, and the vector determination unit 27E may be disposed in the video transmission device 10E or in the remote control device 20E. However, it is preferable that the state identification unit 24E can refer to the recognition result by the object recognition unit 26E without going through the network NW1. It is also preferable that the area identification unit 25E can refer to the identification result by the state identification unit 24E and the determination result by the vector determination unit 27E without going through the network NW1. Therefore, it is preferable that these functional blocks 24E, 25E, 26E, and 27E are all disposed in the video transmission device 10E, or all disposed in the remote control device 20E.

[0148] Also, for example, in the fourth exemplary embodiment, a part or all of the state identification unit 24F, the area identification unit 25F, and the prediction unit 28F may be disposed in the video transmission device 10F or in the remote control device 20F. However, it is preferable that the area identification unit 25F can refer to the identification result by the state identification unit 24F and the prediction result by the prediction unit 28F without going through the network NW1. Therefore, it is preferable that all of these function blocks 24F, 25F, and 28F are disposed in the video transmission device 10F, or all of them are disposed in the remote control device 20F. Furthermore, it is preferable that the state identification unit 24F can refer to the history of the operation information received by the reception unit 23F without going through the network NW1. In this case, it is preferable that all of these function blocks 24F, 25F, and 28F are disposed in the remote control device 20F.

[0149] [Modification 9] In addition, in the video transmission methods S1, S1D, S1E, and S1F executed by each of the exemplary embodiments, the order of performing the steps included in the method is not limited to the order described above. Furthermore, multiple steps included in the method may be performed in parallel. Furthermore, each of the exemplary embodiments may be implemented in combination. Furthermore, each of the functional blocks included in each of the exemplary embodiments is not limited to being consolidated in one device, and may be distributed and arranged in multiple devices. Furthermore, each of the functional blocks included in each of the exemplary embodiments may be arranged in a cloud. Regardless of the device or cloud in which each of these functional blocks is arranged, it is sufficient that the above-mentioned functions of each functional block can be realized by transmitting and receiving information between these functional blocks as necessary.

[0150] [Software implementation example] Some or all of the functions of the video transmission devices 10, 10A, 10B, 10C, 10D, 10E, and 10F may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0151] In the latter case, the video transmission devices 10, 10A, 10B, 10C, 10D, 10E, and 10F are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter, referred to as computer C) is shown in FIG. 20. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the video transmission devices 10, 10A, 10B, 10C, 10D, 10E, and 10F. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the video transmission devices 10, 10A, 10B, 10C, 10D, 10E, and 10F.

[0152] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination of these. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these.

[0153] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may further include a communication interface for transmitting and receiving data to and from other devices. The computer C may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0154] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. Furthermore, the program P can be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

[0155] [Additional Note 1] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiment are also included in the technical scope of the present invention.

[0156] [Additional Note 2] Some or all of the above-described embodiments can be described as follows. However, the present invention is not limited to the following described aspects.

[0157] (Appendix 1) an acquisition means for acquiring an image including, in an angle of view, at least a portion of the work machine and at least a portion of the work area of ​​the work machine; A status identification means for identifying a status of a work performed by the work machine on a work object; an area specifying means for specifying a target area in the video in response to a result of the identification by the state identifying means; a transmitting means for transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image; A video transmission system comprising:

[0158] With the above configuration, it is possible to maintain operability when remotely operating a work machine, while stably transmitting images of the surroundings of the work machine even in an environment where communication quality fluctuates.

[0159] (Appendix 2) The work status identified by the status identification means includes: The state of the working part of the working machine is included. 2. A video transmission system as described in claim 1.

[0160] With the above configuration, the target area can be determined more accurately depending on the state of the work area.

[0161] (Appendix 3) The state of the work site includes: At least one of the position, movement, and presence or absence of the work object at the work site is included. 3. A video transmission system as described in claim 2.

[0162] With the above configuration, the target area can be determined more accurately based on at least one of the position and movement of the work area and the presence or absence of a work object in the work area.

[0163] (Appendix 4) The state specifying means is The state of the work is identified by at least referring to the analysis result of the video. 4. A video transmission system according to any one of claims 1 to 3.

[0164] With the above configuration, the state of work can be determined more accurately based on information obtained from the video.

[0165] (Appendix 5) The state specifying means is Identifying a state of the work by at least referring to information indicating a work phase by the work machine 5. A video transmission system according to any one of claims 1 to 4.

[0166] With the above configuration, the state of work can be determined more accurately based on the information indicating the work phase.

[0167] (Appendix 6) A prediction means for predicting an operation of the work machine is further provided, The area specifying means specifies the target area by further referring to the prediction result by the prediction means. A video transmission system according to any one of claims 1 to 5.

[0168] With the above arrangement, the region of interest can be identified so as to match the region that the operator will be focusing on in response to the predicted operation of the work machine.

[0169] (Appendix 7) The work status identified by the status identification means includes: The status of other work machines located around the work machine is included. 7. A video transmission system according to any one of claims 1 to 6.

[0170] With the above configuration, the target area can be determined with even greater accuracy depending on the state of other work machines.

[0171] (Appendix 8) The acquiring means further acquires communication status information indicating a communication status in a transmission path used by the transmitting means, The transmitting means refers to the communication status information to specify the relative image quality between the target area and an area outside the target area. A video transmission system according to any one of claims 1 to 7.

[0172] With the above configuration, it is possible to flexibly switch, depending on the communication conditions, whether to display at least the target area in adjusted image quality or the entire image in unadjusted image quality at the destination.

[0173] (Appendix 9) an acquisition means for acquiring an image including, in an angle of view, at least a portion of the work machine and at least a portion of the work area of ​​the work machine; A status identification means for identifying a status of a work performed by the work machine on a work object; an area specifying means for specifying a target area in the video in response to a result of the identification by the state identifying means; a transmitting means for transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image; A video transmitting device comprising:

[0174] According to the above configuration, the same effects as those of Supplementary Note 1 are achieved.

[0175] (Appendix 10) The work status identified by the status identification means includes: The state of the working part of the working machine is included. 10. A video transmission device as described in claim 9.

[0176] According to the above configuration, the same effects as those of Supplementary Note 2 are achieved.

[0177] (Appendix 11) The state of the work site includes: At least one of the position, movement, and presence or absence of the work object at the work site is included. 11. A video transmission device as described in claim 10.

[0178] According to the above configuration, the same effects as those of Supplementary Note 3 are achieved.

[0179] (Appendix 12) The state specifying means is The state of the work is identified by at least referring to the analysis result of the video. 12. A video transmitting device according to any one of appendixes 9 to 11.

[0180] According to the above configuration, the same effects as those of Supplementary Note 4 are achieved.

[0181] (Appendix 13) The state specifying means is Identifying a state of the work by at least referring to information indicating a work phase by the work machine 13. A video transmitting device according to any one of claims 9 to 12.

[0182] According to the above configuration, the same effects as those of Supplementary Note 5 are achieved.

[0183] (Appendix 14) A prediction means for predicting an operation of the work machine is further provided, The area specifying means specifies the target area by further referring to the prediction result by the prediction means. 14. A video transmission device according to any one of claims 9 to 13.

[0184] According to the above configuration, the same effect as that of Supplementary Note 6 is achieved.

[0185] (Appendix 15) The work status identified by the status identification means includes: The status of other work machines located around the work machine is included. 15. A video transmitting device according to any one of claims 9 to 14.

[0186] According to the above configuration, the same effects as those of Supplementary Note 7 are achieved.

[0187] (Appendix 16) The acquiring means further acquires communication status information indicating a communication status in a transmission path used by the transmitting means, The transmitting means refers to the communication status information to specify the relative image quality between the target area and an area outside the target area. 16. A video transmission device according to any one of claims 9 to 15.

[0188] According to the above configuration, the same effects as those of Supplementary Note 8 are achieved.

[0189] (Appendix 17) acquiring an image including, within an angle of view, at least a portion of the work machine and at least a portion of a work area of ​​the work machine; Identifying a state of a work performed by the work machine on a work object; Identifying a target area in the video according to the result of identifying the state of the work; Transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image; A video transmission method comprising:

[0190] According to the above configuration, the same effects as those of Supplementary Note 1 are achieved.

[0191] (Appendix 18) The specified work state includes: The state of the working part of the working machine is included. 18. The video transmission method of claim 17.

[0192] According to the above configuration, the same effects as those of Supplementary Note 2 are achieved.

[0193] (Appendix 19) The state of the work site includes: At least one of the position, movement, and presence or absence of the work object at the work site is included. 19. The video transmission method of claim 18.

[0194] According to the above configuration, the same effects as those of Supplementary Note 3 are achieved.

[0195] (Appendix 20) When identifying the status of the operation, Refer to at least the analysis results of the video. 20. A video transmission method according to any one of appendix 17 to 19.

[0196] According to the above configuration, the same effects as those of Supplementary Note 4 are achieved.

[0197] (Appendix 21) When identifying the status of the operation, At least information indicating a work phase by the work machine is referenced. 21. A video transmission method according to any one of appendix 17 to 20.

[0198] According to the above configuration, the same effects as those of Supplementary Note 5 are achieved.

[0199] (Appendix 22) predicting the operation of the work machine; When identifying the region of interest, Further referring to the predicted results of the operation of the work machine 22. A video transmission method according to any one of appendix 17 to 21.

[0200] According to the above configuration, the same effect as that of Supplementary Note 6 is achieved.

[0201] (Appendix 23) The specified work state includes: The status of other work machines located around the work machine is included. 23. A video transmission method according to any one of appendix 17 to 22.

[0202] According to the above configuration, the same effects as those of Supplementary Note 7 are achieved.

[0203] (Appendix 24) Further acquiring communication status information indicating a communication status in a transmission path for transmitting the adjusted image; By referring to the communication status information, the relative image quality of the target area and the outside of the target area is identified. 24. A video transmission method according to any one of appendix 17 to 23.

[0204] According to the above configuration, the same effects as those of Supplementary Note 8 are achieved.

[0205] (Appendix 25) A program for causing a computer to function as a video transmission device, the program comprising: an acquisition means for acquiring an image including, in an angle of view, at least a portion of the work machine and at least a portion of the work area of ​​the work machine; A status identification means for identifying a status of a work performed by the work machine on a work object; an area specifying means for specifying a target area in the video in response to a result of the identification by the state identifying means; a transmitting means for transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image; A program that functions as a

[0206] According to the above configuration, the same effects as those of Supplementary Note 1 are achieved.

[0207] (Appendix 26) The work status identified by the status identification means includes: The state of the working part of the working machine is included. 26. The program according to claim 25.

[0208] According to the above configuration, the same effects as those of Supplementary Note 2 are achieved.

[0209] (Appendix 27) The state of the work site includes: At least one of the position, movement, and presence or absence of the work object at the work site is included. 27. A video transmission system as described in claim 26.

[0210] According to the above configuration, the same effects as those of Supplementary Note 3 are achieved.

[0211] (Appendix 28) The state specifying means is The state of the work is identified by at least referring to the analysis result of the video. 28. The program of any one of appendixes 25 to 27.

[0212] According to the above configuration, the same effects as those of Supplementary Note 4 are achieved.

[0213] (Appendix 29) The state specifying means is Identifying a state of the work by at least referring to information indicating a work phase by the work machine 29. The program according to any one of appendices 25 to 28.

[0214] According to the above configuration, the same effects as those of Supplementary Note 5 are achieved.

[0215] (Appendix 30) A prediction means for predicting an operation of the work machine is further provided, The area specifying means specifies the target area by further referring to the prediction result by the prediction means. 30. The program according to any one of appendix 25 to 29.

[0216] According to the above configuration, the same effect as that of Supplementary Note 6 is achieved.

[0217] (Appendix 31) The work status identified by the status identification means includes: The status of other work machines located around the work machine is included. A program according to any one of appendix 25 to 30.

[0218] According to the above configuration, the same effects as those of Supplementary Note 7 are achieved.

[0219] (Appendix 32) The acquiring means further acquires communication status information indicating a communication status in a transmission path used by the transmitting means, The transmitting means refers to the communication status information to specify the relative image quality between the target area and an area outside the target area. 32. The program according to any one of appendix 25 to 31.

[0220] According to the above configuration, the same effects as those of Supplementary Note 8 are achieved.

[0221] [Additional Note 3] A part or all of the above-described embodiments can be further expressed as follows.

[0222] At least one processor, the processor comprising: an acquisition process for acquiring an image including, in an angle of view, at least a part of the work machine and at least a part of the work area of ​​the work machine; A status identification process for identifying a status of a work performed by the work machine on a work object; an area identification process for identifying a target area in the video in accordance with a result of the state identification process; and a transmission process for transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image.

[0223] The video transmission device may further include a memory, and the memory may store a program for causing the processor to execute the acquisition process, the state identification process, the area identification process, and the transmission process. The program may be recorded in a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0224] 1, 1A, 1B, 1C, 1D, 1E, 1F Video transmission system 2F, 10D, 12, 12D, 24E, 24F Status Specifications 10, 10A, 10B, 10C, 10D, 10E, 10F Video transmitting device 60C Control Unit 11, 11D, 11E, 11F acquisition section 12, 12D, 24E, 24F Status Specific Part 13, 13D, 25E, 25F area identification part 14, 14D, 14E, 14F Transmitter 15D, 15E, 15F adjustment section 26E Object recognition section 27E Vector judgment unit 28F Prediction Department 20, 20A, 20B, 20C, 20D, 20E, 20F Remote control device 21D, 21E, 21F Decoding section 22D, 22E, 22F display section 23D, 23E, 23F Reception

Claims

1. an acquisition means for acquiring an image including, in an angle of view, at least a portion of the work machine and at least a portion of the work area of ​​the work machine; A status identification means for identifying a status of a work performed by the work machine on a work object; an area specifying means for specifying a target area in the video in response to a result of the identification by the state identifying means; a transmitting means for transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image, the acquiring means, the area identifying means, and the transmitting means are included in a video transmitting device, The state specifying means is included in a remote control device, The transmission means transmits the adjusted image to the remote control device, The state identification means transmits the state of the work identified based on the received adjusted image to the image transmission device, The area identification means identifies a new target area in the new image acquired by the acquisition means based on the received work status. Video transmission system.

2. The work status identified by the status identification means includes: The state of the working part of the working machine is included. The video transmission system according to claim 1 .

3. The state specifying means is The state of the work is identified by at least referring to the analysis result of the video.

3. A video transmission system according to claim 1 or 2.

4. The state specifying means is Identifying a state of the work by at least referring to information indicating a work phase by the work machine The video transmission system according to any one of claims 1 to 3.

5. A prediction means for predicting an operation of the work machine is further provided, The area specifying means specifies the target area by further referring to the prediction result by the prediction means. The video transmission system according to any one of claims 1 to 4.

6. The work status identified by the status identification means includes: The status of other work machines located around the work machine is included. A video transmission system according to any one of claims 1 to 5.

7. The acquiring means further acquires communication status information indicating a communication status in a transmission path used by the transmitting means, The transmitting means refers to the communication status information to specify the relative image quality between the target area and an area outside the target area. The video transmission system according to any one of claims 1 to 6.

8. an acquisition means for acquiring an image including, in an angle of view, at least a portion of the work machine and at least a portion of the work area of ​​the work machine; an area specifying means for specifying a target area in the image in accordance with a result of specifying a state of work performed by the work machine on a work object; a transmitting means for transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image to a remote control device; Including, The area specifying means receives, from the remote control device, a state of the work specified based on the adjusted image in the remote control device, and specifies a new target area in the new image acquired by the acquisition means based on the received state of the work. Video transmission device.

9. acquiring an image including, within an angle of view, at least a portion of the work machine and at least a portion of a work area of ​​the work machine; Identifying a state of a work performed by the work machine on a work object; Identifying a target area in the video according to the result of identifying the state of the work; Transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image; Including, the acquiring, the identifying, and the transmitting are performed by a video transmitting device; The determining of the status of the operation is performed by a remote control device; In the transmitting, the video transmission device transmits the adjusted video to the remote control device, In the step of determining the status of the work, the remote control device transmits the status of the work determined based on the received adjusted image to the image transmission device; In identifying the target area, the video transmission device identifies a new target area in the new video acquired by the acquisition based on the received status of the work. Video transmission method.

10. A program for causing a computer to function as a video transmission device, the program comprising: an acquisition means for acquiring an image including, in an angle of view, at least a portion of the work machine and at least a portion of the work area of ​​the work machine; an area specifying means for specifying a target area in the image in accordance with a result of specifying a state of work performed by the work machine on a work object; a transmitting means for transmitting an adjusted image obtained by adjusting the relative image quality of the target area and an area outside the target area in the image to a remote control device; Function as a The area specifying means receives, from the remote control device, a state of the work specified based on the adjusted image in the remote control device, and specifies a new target area in the new image acquired by the acquisition means based on the received state of the work. program.

Images