Work machine video transmission system

Abstract

To provide an image transmission system for work machines in which remote control and surrounding monitoring of each work machine are unlikely to be hindered even when a plurality of work machines transmit camera image data to a remote location.SOLUTION: An image management server 200 determines priorities of a plurality of on-vehicle controllers 300 based on working states of a plurality of work machines 1 and a priority table 502, calculates an upper limit value of a transmission data amount of each on-vehicle controller based on the priorities of the plurality of on-vehicle controllers so that a total data amount of image data of a plurality of cameras 310 transmitted from the plurality of on-vehicle controllers does not exceed a predetermined value, and sends the calculated upper limit value of a transmission data amount to each on-vehicle controller. Each on-vehicle controller adjusts image quality of the image data of the plurality of cameras so that a data amount does not exceed the upper limit value of the transmission data amount transmitted from the image management server, and transmits the image data after image quality adjustment to a management computer.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a video transmission system for a work machine. [Background technology]

[0002] In recent years, video transmission systems have been used in which multiple cameras are mounted on construction machinery such as hydraulic excavators, and the images of the surrounding area captured by the multiple cameras are sent via a network to a remote terminal and displayed on a monitor. This system can be used for remote control of construction machinery and surrounding monitoring.

[0003] In this type of video transmission system, if the amount of camera video data sent from the work machine (camera) to the remote terminal (monitor) increases and reaches the upper limit of the data communication volume on the communication path, the image on the remote monitor may become distorted (including insufficient resolution) or may be missing. For example, when a hydraulic excavator rotates, the subject being photographed by the camera mounted on the rotating body moves significantly, which increases the amount of video data transmitted and may cause distorted or missing images on the monitor.

[0004] In order to solve this type of problem, Patent Document 1 discloses a video transmission system in which multiple cameras that capture surveillance footage of the surroundings of a construction machine, an operating lever that remotely controls the construction machine, and a display (monitor) that displays the surveillance footage of the surroundings are connected so that they can communicate with each other via a network, and the video transmission system is equipped with an encoder and a decoder arranged on the network, a connection / disconnection member (switch) that switches between connecting and disconnecting communication between the encoder and the decoder, and a control device that, when an operation signal of the operating lever corresponding to the target speed of the actuator of the construction machine is above an allowable judgment threshold, operates the connection / disconnection member so as to connect only the encoder and decoder that are connected to the display (monitor) that displays surveillance footage of the surroundings in the direction in which the construction machine is driven by the operation signal. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2017-092908 Summary of the Invention [Problem to be solved by the invention]

[0006] That is, in Patent Document 1, of the multiple camera images mounted on a single construction machine, the camera images to be transmitted to a decoder via a communication line and the camera images not to be transmitted are selected in accordance with the operation of the operator. For example, when a right turn operation is input to the control lever, only the camera image showing the right side of the rotating body is transmitted from the construction machine to the communication line, and other camera images (for example, camera images showing the left side of the rotating body) are not transmitted from the construction machine to the communication line. This makes it possible to maintain the quality of the image showing the right side of the rotating body, which is necessary when turning right.

[0007] However, simply limiting the camera footage transmitted within a single construction machine may not be enough to avoid straining the communication line capacity. In other words, even if the amount of data transmitted for the camera footage of that construction machine can be reduced, if the amount of data transmitted for the camera footage of other construction machines using the same network (for example, construction machines operating at the same construction site) is large, there is still a risk that the camera footage of that construction machine will be distorted or missing due to the upper limit of the communication capacity of that network.

[0008] The present invention has been made in consideration of the above circumstances, and its object is to provide a work machine video transmission system that is less likely to interfere with the remote control of each work machine or the monitoring of the surroundings, even when multiple work machines transmit camera video data to remote locations. [Means for solving the problem]

[0009] The present application includes a plurality of means for solving the above-mentioned problems, and one example thereof is a video transmission system for work machines comprising a plurality of on-board controllers that are mounted on a plurality of work machines each having a plurality of cameras and that transmit video data from the plurality of cameras to a network, an image management server connected to the network so as to be able to communicate with the plurality of on-board controllers, and a computer connected to the network so as to be able to communicate with the plurality of on-board controllers, that receives the video data from the plurality of cameras transmitted from the plurality of on-board controllers and displays it on a monitor, wherein the image management server is configured to manage the work status of the plurality of work machines, Priority is set in advance for each operating condition of the work machine. The priority of the plurality of on-board controllers is determined based on the correspondence between the work status of the work machine and the priority, an upper limit value for the transmission data amount of each on-board controller is calculated based on the priority of the plurality of on-board controllers so that the total data amount of the video data of the plurality of cameras transmitted from the plurality of on-board controllers to the computer is within a predetermined value, and the calculated upper limit value for the transmission data amount is transmitted to each on-board controller, and the plurality of on-board controllers each adjust the image quality of the video data of the plurality of cameras so that the image quality transmitted from the image management server is within the upper limit value for the transmission data amount, and transmit the video data after the image quality adjustment to the computer. The plurality of work machines are a plurality of hydraulic excavators, and the work states of the work machines include "loading," which indicates that the hydraulic excavator is loading the bed of a dump truck, and "digging," which indicates that the hydraulic excavator is excavating, and the priority defined by the correspondence between the work state of the work machine and the priority is set to be the highest for "loading," and the image management server calculates the upper limit value of the transmission data amount so that the upper limit value of the transmission data amount increases for an on-board controller with a relatively higher priority. It was decided that. [Effects of the Invention]

[0010] According to the present invention, the priority of the amount of video data to be transmitted is changed depending on the work status of the work machine, and the number and quality of video data according to the work being performed by the work machine is transmitted to the management computer, thereby reducing the possibility of interference with remote control of the work machine or surrounding monitoring. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing the overall configuration of a video transmission system according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram of the image management server 200. [Figure 3]FIG. 1 is a schematic configuration diagram of a management computer 100. [Figure 4] FIG. 2 is a schematic configuration diagram of an in-vehicle controller 300. [Figure 5] FIG. 3 is a diagram showing an example of part of the data stored in the storage device 312 of the in-vehicle controller 300A in a table format. [Figure 6] FIG. 2 is a diagram showing in table form some of the data stored in the storage device 212 of the image management server 200. [Figure 7] FIG. 10 is a diagram showing an example of part of the data stored in the storage device 312 of the in-vehicle controller 300B in a table format. [Figure 8] FIG. 10 is a diagram showing an example of part of the data stored in the storage device 312 of the in-vehicle controller 300C in a table format. [Figure 9] 10 is a flowchart of a process executed by the image management server 200. [Figure 10] FIG. 10 is a diagram showing an example of an equation used to calculate an upper limit of transmission data. [Figure 11] 4 is a flowchart of a process executed by each in-vehicle controller 300 according to the first embodiment. [Figure 12] FIG. 10 is a diagram showing the overall configuration of a video transmission system according to a second embodiment of the present invention. [Figure 13] FIG. 10 is a diagram showing, in table format, part of the data stored in the storage device 212 of the image management server 200 according to the second embodiment. [Figure 14] FIG. 10 is an explanatory diagram for setting the working status of a hydraulic excavator to "digging" or "loading" using position data of the hydraulic excavator and dump truck. [Figure 15] 10 is a flowchart of a process executed by the image management server 200 of the second embodiment. [Figure 16] 16 is a detailed flowchart of the process performed in S1012 of FIG. 15. [Figure 17] 10 is a flowchart of a process executed by each in-vehicle controller 300 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] In the following description, when there are multiple identical components, the reference numeral may be suffixed with an uppercase or lowercase letter, but the multiple components may be collectively referred to without the uppercase or lowercase letter. For example, when there are three identical computers 112A, 112B, and 112C, they may be collectively referred to as computer 112.

[0014] First Embodiment Fig. 1 is a diagram showing the overall configuration of a video transmission system according to a first embodiment of the present invention. The video transmission system in Fig. 1 includes a plurality of on-board controllers 300A, 300B, 300C, and 300D mounted, for example, one on each of a plurality of work machines (hydraulic excavators in the example of Fig. 1) 1A, 1B, 1C, and 1D operating at the same work site, an image management server 200, and at least one management computer (referred to as "management PC" in Fig. 1) 100A, 100B, 100C, and 100D.

[0015] The image management server 200 calculates an upper limit value for the amount of transmission data (transmission data upper limit 406 (see Figure 6)) for each vehicle controller 300 so that the total amount of video data from multiple cameras 310 (see Figure 4) transmitted from multiple vehicle controllers 300 to the management computer 100 is within a predetermined value, and transmits the calculated upper limit value for the amount of transmission data (transmission data upper limit 406) to each vehicle controller 300.

[0016] Each vehicle controller 300 adjusts the image quality of the video data from multiple cameras 310 (see Figure 4) so ​​that the amount of data sent from the image management server 200 is within the upper limit (upper limit 406 (see Figures 5, 7, and 8)), and sends the video data after the image quality adjustment to the corresponding management computer 100.

[0017] The management computer 100 displays the image data (camera image) after image quality adjustment transmitted from the on-board controller 300 on the monitor 120 (see FIG. 3), and provides the operator in front of the monitor 120 with an image of the situation around the work machine 1.

[0018] Each on-board controller 300 is connected to a wide area network (WAN) via the nearest base stations 50A, 50B using, for example, a communication standard line for mobile phones. The base stations 50A, 50B are the first base stations through which data transmitted from multiple on-board controllers 300 (including video data from the cameras 310) passes along the communication path (WAN) from the on-board controllers 300 to the management computer 100. Because radio wave conditions may vary depending on the location of the work machine 1, each work machine 1 (on-board controller 300) may connect to a different base station 50. In the example of FIG. 1, three work machines 1A, 1B, and 1C (on-board controllers 300A, 300B, and 300C) are connected to base station 50A, and work machine 1D (on-board controller 300D) is connected to base station 50B.

[0019] The WAN is further connected to an image management server 200 and management computers 100A, 100B, 100C, and 100D, and terminals 100, 200, and 300 connected to the WAN are capable of mutual data communication. The WAN may include the Internet 10, which is a public line network, as shown in Figure 1, or may be composed of only dedicated lines.

[0020] Furthermore, each on-board controller 300 and the image management server 200 may be connected to the same local area network (LAN), for example, via Wi-Fi, as shown in Fig. 1. Using a LAN connection makes it possible to easily identify work machines 1 that are located near the image management server 200 (for example, at the same work site).

[0021] Note that because the terminals 100, 200, 300 are connected to a WAN, the use of a LAN is not essential and only the WAN may be used. When only the WAN is used, a work machine 1 that is present at the same work site as the image management server 200 can be identified by, for example, comparing position data from a GNSS receiver mounted on the work machine 1 with map data that includes position data of the work site where the image management server 200 is installed.

[0022] (Configuration of image management server 200) 2 is a schematic diagram of the image management server 200. The image management server 200 is a server that includes a processor such as a CPU 211, a storage device 212 such as a ROM, RAM, flash memory, or hard disk drive that can store programs and various data that can be executed by the processor, and a communication device 215 that is connected to a WAN and a LAN. The image management server 200 calculates an upper limit value for the amount of data that can be transmitted to each on-board controller 300, and transmits the calculated upper limit value for the amount of data that can be transmitted to the target on-board controller 300. The communication device 215 can be connected to both a LAN and a WAN, and different communication devices may be used for the LAN and the WAN.

[0023] (Configuration of management computer 100) 3 is a schematic diagram of each management computer 100. Each management computer 100 is a computer (e.g., a personal computer) that includes a processor such as a CPU 111, a storage device 112 such as a ROM, RAM, flash memory, or hard disk drive that can store programs and various data that can be executed by the processor, and a communication device 115 connected to a WAN. Connected to the management computer 100 are a monitor 120 that displays camera video data (described below) transmitted from the corresponding on-board controller 300, and an input device 130 (e.g., a mouse, keyboard, touch panel, etc.) for providing data, instructions, etc. to the management computer 100.

[0024] When the management computer 100 is used for remotely controlling each work machine 1, an operation device (e.g., an operation lever) 150 may be connected to output an operation signal for the work machine 1 (on-board controller 300) to be remotely controlled in response to an input operation. In this embodiment, the operation device 150 is connected to the management computer 100 because the management computer 100 is used for remote control. In this embodiment, there is a one-to-one correspondence between one management computer 100 and one on-board controller 300, and those with the same alphabet (uppercase letter) at the end of their reference numerals correspond to each other. For example, when the operation device 150 of the management computer 100A is operated, the work machine 1A equipped with the corresponding on-board controller 300A operates in accordance with the operation of the operation device 150.

[0025] (Configuration of the in-vehicle controller 300) 4 is a schematic diagram of each on-board controller 300. Each on-board controller 300 includes a processor such as a CPU 311, a storage device 312 such as a ROM, RAM, flash memory, or hard disk drive in which programs that can be executed by the processor and various data can be stored, and a communication device 315 connected to a WAN and a LAN.

[0026] In addition, the on-board controller 300 is connected via a controller area network (CAN) to, for example, a plurality of cameras 310a, 310b, 310c mounted on the work machine 1, an engine 305 that drives a hydraulic pump that is the hydraulic supply source to the hydraulic actuator mounted on the work machine 1, a rotation speed sensor 306 for detecting the rotation speed of the engine 305 (engine rotation speed), a plurality of solenoid valves (solenoid proportional valves) 302 that generate pilot pressure for a control valve that controls the hydraulic oil supplied to and discharged from the hydraulic actuator, and a plurality of pressure sensors 303 that detect the pilot pressure generated by the plurality of solenoid valves 302.

[0027] Multiple cameras 310a, 310b, 310c are mounted on the work machine 1 and capture images of the surroundings of the work machine 1. The image data captured by each camera 310 is first stored in a storage device 312 in an on-board controller, for example, via a CAN, and then transmitted to a predetermined management computer 100 via a WAN. In this embodiment, the cameras 310 are mounted on the rotating body of each hydraulic excavator (work machine) 1, and each captures images in a different direction (for example, the front, right side, or left side of the rotating body). The management computer 100 displays the received camera image data on the monitor 120. There is no particular limit to the number of cameras 310 that can be mounted on a single work machine 1, and four or more cameras may be mounted.

[0028] When the management computer 100 is used to remotely control the work machine 1, there is a one-to-one correspondence between the on-board controller 300 and the management computer 100, and an operator seated in front of the monitor 120 operates the operation device 150 connected to the management computer 100 while watching the image (camera image) on the monitor 120 to operate the work machine 1. When the operation device 150 is operated, an operation signal corresponding to the operation is transmitted from the management computer 100 to the corresponding on-board controller 300 via a WAN. The on-board controller 300 receives the operation signal and operates the solenoid valve 302 corresponding to the operation signal based on the operation signal. This drives the control valve, operates the hydraulic actuator, and remote control of the work machine 1 by the operator is realized.

[0029] (Data from the 300A in-vehicle controller) 5 is a diagram showing an example of a table of some of the data stored in the storage device 312 of the on-board controller 300A. As shown in this diagram, the storage device 312 of each on-board controller 300 stores work status data 401 indicating the work status of the work machine on which it is mounted (hereinafter sometimes referred to as "own device"), first camera image quality data 402 indicating the image quality of the video data from the first camera 310a of the own device to be transmitted to the WAN, second camera image quality data 403 indicating the image quality of the video data from the second camera 310b of the own device to be transmitted to the WAN, third camera image quality data 404 indicating the image quality of the video data from the third camera 310c of the own device to be transmitted to the WAN, connection destination data 405 indicating the base station to which the own device is connected while using the WAN, and transmission data upper limit data 406 indicating the upper limit of the amount of data that the own device will transmit to the WAN.

[0030] Examples of work status data 401 include "loading", which indicates that the vehicle is currently loading soil and sand into the bed of a dump truck; "digging", which indicates that the vehicle is currently digging soil and sand with its front work implement; "leveling", which indicates that the vehicle is currently leveling the construction surface (finishing work); "cleaning", which indicates that the vehicle is currently cleaning its body; and "moving to garage", which indicates that the vehicle has completed its scheduled work and is currently being moved to the garage.

[0031] In this embodiment, the work status data 401 is set by the operator of the management computer 100 (in this embodiment, the operator who remotely controls the work machine 1 using the operation device 150). If the operator has not set the work status after the on-board controller 300 has been started up, the work status is maintained at the initial value of "not specified" (see FIG. 6).

[0032] The image quality data 402, 403, 404 from each camera is preset to multiple image quality grades, such as high quality, medium quality, and low quality. The data volume (transfer speed) for each image quality is roughly 100Mbps for high quality, 50Mbps for medium quality, and 10Mbps for low quality. Note that in this paper, the image quality of each piece of video data is explained as indicating the image quality at the time of shooting, but it is also possible to shoot at a uniform high image quality during shooting, and then compress the data to a lower image quality as needed when transmitting it over a WAN.

[0033] The connection destination data 405 may store, for example, the identifier of the currently connected base station, such as the first base station 50A or the second base station 50B shown in FIG. 1. Generally, there are multiple base stations near one work site. Since radio wave conditions vary depending on the location of the work machine at the work site, each work machine may connect to a different base station in order to find a base station with better radio wave conditions.

[0034] The upper limit of the data transmission speed can be set as the transmission data upper limit data 406 when the device itself transmits video data from multiple cameras 310 to the WAN. For example, the transmission data upper limit of the device itself in Fig. 5 is "500Mbps", which indicates that the device itself can use the WAN up to a maximum of 500Mbps.

[0035] (Image management server 200 data) 6 is a diagram showing in table form some of the data stored in the storage device 212 of the image management server 200. The storage device 212 of the image management server 200 stores a vehicle information table 501 that compiles information on multiple work machines 1 (for example, work machines working at the same work site) for which the image management server 200 manages transmission data upper limits, and a priority table 502 that predefines the correspondence between the work status of the work machine 1 and the priority of transmission data.

[0036] The priority table 502 is a table into which a preset priority is input for each work state of the work machine 1. In other words, the priority table 502 defines the correspondence between the work state of the work machine 1 and priority. "Priority" indicates the priority of the amount of data that each on-board controller 300 can transmit to the WAN. In this embodiment, the higher the priority of the work state, the higher the value in the table (priority score 550) is set, and as a result, the higher the priority score of the on-board controller 300 (work machine 1), the relatively larger the upper limit value of the amount of data that can be transmitted. Note that the correspondence between the work state and priority may be defined by something other than a table.

[0037] 6, the priority score is set to 5 for "loading," 3 for "digging" and "leveling," 2 for "cleaning" and "moving to garage," and 1 for "no designation" when no work state is specified ("no designation"). In other words, in this embodiment, the priority score is set to the highest for "loading" (loading work), which primarily operates the front work implement and upper swing body in cooperation with the dump truck; the priority scores for "digging" and "leveling," which primarily operate the front work implement; the priority scores for "cleaning" and "moving to garage," which do not actively operate the front work implement, are set to the third highest; and the priority score for "no designation" is set to the lowest value.

[0038] The vehicle body information table 501 stores work status data 401, connection destination data 405, priority score data 550, and transmission data upper limit data 406 for each work machine (vehicle body 1A-vehicle body 1D) managed by the image management server 200. The image management server 200 receives the work status data 401 and connection destination data 405 from each on-board controller 300 and stores them in the vehicle body information table 501.

[0039] The priority score data 550 is input with a priority score determined by the image management server 200 based on the work status data 401 acquired from each on-board controller 300 and the priority table 502. For example, since the work status of vehicle body 1A is "loading" and the priority score for "loading" in the priority table 502 is "5," the image management server 200 inputs "5" as the priority score data 550 for vehicle body 1A.

[0040] When loading work is performed by remote control, the operator needs to accurately view the position of the bucket tip of the hydraulic excavator 1 and the position of the dump truck via the monitor 120, which requires high-resolution video data from the camera 310. In the case of Figure 6, the upper limit of transmission data for vehicle body 1A during "loading" is "500 Mbps," which is set higher than the upper limit of transmission data for vehicle body 1B (300 Mbps) and vehicle body 1C (200 Mbps), and therefore the work machine can be operated with higher resolution.

[0041] The details of how the image management server 200 calculates the transmission data upper limit data 406 will be explained later with reference to FIG.

[0042] (Data from the on-board controllers 300B and 300C) Fig. 7 is a diagram showing an example of a table showing some of the data stored in the storage device 312 of the on-board controller 300B. The table items are the same as those of the on-board controller 300A in Fig. 5, but in this example, the work status data 401 of the on-board controller 300B (work machine 1B) is set to "digging" and the transmission data upper limit data 406 is set to "300Mbps".

[0043] Figure 8 is a diagram showing an example of some of the data stored in the storage device 312 of the on-board controller 300C in table format. The table items are the same as those for the on-board controller 300A in Figure 5, but in this example, the work status data 401 for the on-board controller 300C (work machine 1C) is set to "cleaning", and the transmission data upper limit data 406 is set to "200Mbps". Note that "camera off" has been entered as the image quality data for the third camera 310c, which indicates that the capture of video data around the work machine by the third camera 310c has been suspended.

[0044] (Processing of image management server 200) Next, the process by which the image management server 200 sets the transmission data upper limit data 406 of each vehicle controller 300A-300C to the value shown in Figures 5, 7, and 8 when the working status data 401 and connection destination data 405 of the vehicle controllers 300A-C are as shown in Figures 5, 7, and 8 will be explained using a flowchart (Figure 9) of the processing executed by the image management server 200.

[0045] Figure 9 is a flowchart of the processing executed by the image management server 200. The image management server 200 starts the processing of the flowchart in Figure 9 at a predetermined cycle. When processing starts, the image management server 200 collects information about the vehicle bodies (work machines) 1 that it is managing. Specifically, in S1001, the image management server 200 first acquires the identification information of the work machines 1A-1D connected to the same LAN, creates columns in the vehicle body information table 501 for the number of work machines 1A-1D, and assigns the work machines (vehicle bodies) 1A-1D to each column. Note that the on-board controllers 300A-300D may also be assigned to each column.

[0046] In S1002, the image management server 200 requests each managed work machine 1A-1D (each on-board controller 300A-300D) to send work status data 401, and then stores the work status data 401 of each vehicle body (each work machine) 1A-1D sent from each on-board controller 300A-300D in the vehicle body information table 501.

[0047] In S1003, the image management server 200 requests each work machine 1A-1D (each on-board controller 300A-300D) to be managed to send connection destination data 405, and then stores the connection destination data 405 of each vehicle body (each work machine) 1A-1D sent from each on-board controller 300A-300D in the vehicle body information table 501.

[0048] In S1101, the image management server 200 extracts from the priority table 502 the priority score corresponding to the work status data 401 acquired in S1002, and stores the extracted priority score in the priority score data 550 of the corresponding vehicle body (work machine) 1A-1D in the vehicle body information table 501. In other words, the priority score becomes the priority of the corresponding on-board controller 300. For example, since the work status of the on-board controller 300A is "loading" as shown in FIG. 5, a priority score of "5" is entered as the priority score data 550 of the vehicle body 1A in the vehicle body information table 501 (see FIG. 6). Similarly, priority score data 550 is extracted for the other vehicle bodies 1B-1D based on the work status data 401 and the priority table 502, and the extracted priority score data 550 is entered into the vehicle body information table 501.

[0049] In S1102, the image management server 200 references the connection destination data 405 of each vehicle body 1A-1D in the vehicle body information table 501 to determine whether there are multiple vehicle bodies 1 that are connected to the same base station 50. If there are multiple work machines 1 that are connected to the same base station 50, the image management server 200 calculates the transmission data upper limit data 406 for each of the multiple work machines 1 based on the work status data 401 and priority score data 405 of the multiple work machines 1.

[0050] The reason for this calculation is that there is an upper limit to the amount of data that can be exchanged by one base station 50. In the case of Figure 6, the data volume of base station 50A is shared among three vehicles (work machines) 1A, 1B, 1C, so adjustment of the transmission data upper limit is only performed for the three vehicles (work machines) 1A, 1B, 1C that are connected to the same "first base station 50A" as their destination.

[0051] On the other hand, if there are not multiple work machines 1 connected to the same base station 50, there is no need to impose a limit on the amount of data that can be transmitted, so the transmission data upper limit data 406 of the work machine 1 connected to that base station 50 is not calculated. In the case of Figure 6, only the vehicle body 1D is connected to the "second base station 50B", and there is no need to adjust the amount of data that can be transmitted.

[0052] When calculating the transmission data upper limit data 406, the image management server 200 uses formula (1) shown in Fig. 10. The denominator of formula (1) is the sum of the priority scores of the vehicles to be managed, which in the above example is the sum of the priority scores of the three vehicles 1A-1C (10). The numerator of formula (1) is the product of the priority score of the work machine (own machine) that is the calculation target of the "transmission data upper limit" and a predetermined transmission data volume (1000Mbps), which in the above example is the value obtained by multiplying any of the priority scores 550 of the vehicles 1A-1C by 1000Mbps. It should be noted that the "predetermined transmission data volume" is preferably set to a value equal to or less than the upper limit of the transmission data volume that the target base station 50 can handle, and in the above example it is set to 1000Mbps, which is the upper limit of the transmission data volume of base station 50A. In this case, the transmission data upper limit 406 for vehicle body (work machine) 1A is 500Mbps, the transmission data upper limit 406 for vehicle body (work machine) 1B is 300Mbps, and the transmission data upper limit 406 for vehicle body (work machine) 1C is 200Mbps. As a result, the upper limit value of the transmission data amount is calculated to be larger for on-vehicle controllers 300A-300C (vehicle bodies 1A-1C) with relatively higher priority scores, and transmission data upper limit 406 can be allocated to each on-vehicle controller 300A-300C within the range of the upper limit value of the transmission data amount of first base station 50A.

[0053] Note that formula (1) in Figure 10 is just one example of a formula for calculating the transmission data upper limit 406, and any formula can be used as long as it is within the range of the upper limit of the amount of data that a certain base station 50 can process, and the higher the priority score of the in-vehicle controller 300, the larger the transmission data upper limit 406.

[0054] When the calculation of the transmission data upper limit 406 is completed, in S1103 the image management server 200 transmits the transmission data upper limit data 406 to each of the vehicle bodies 1A-1C (vehicle controllers 300A-300C) whose transmission data upper limits were calculated in S1102.

[0055] (Processing of the in-vehicle controller 300) Next, we will explain the flow of processing executed by each on-board controller 300. Fig. 11 shows a flowchart of processing executed by each on-board controller 300 according to this embodiment. When the power supply of the work machine 1 is turned on from OFF to ON, the on-board controller 300 starts up and starts the processing of the flowchart in Fig. 11.

[0056] First, in S2001, the in-vehicle controller 300 initializes the tables in the storage device 312 shown in FIGS. 5, 7, and 8. As a result, each piece of data 401-406 in the tables in the storage device 312 is set to its initial value. The initial value of the work status data 401 is "not specified." The initial values ​​of the image quality data 402, 403, and 404 of the first to third cameras are all set to "medium image quality," the initial value of the connection destination data 405 is set to "none," and the initial value of the transmission data upper limit 406 is set to "100 Mbps." Note that the initial values ​​of the image quality data 402, 403, and 404 of the first to third cameras and the transmission data upper limit 406 may be set to other values.

[0057] Next, in S2002, the on-board controller 300 performs processing to set work status data 401 that indicates the work status of its own machine. In this embodiment, the setting processing is performed by the on-board controller 300 accepting input of work status data 401 from the operator of the corresponding management computer 100 via the operation device 150. The operator of the management computer 100 operates the operation device 150 to input the work status of the work machine 1 to be remotely controlled into the management computer 100 while viewing the video data from the camera 310 displayed on the monitor 120. The management computer 100 generates work status data 401 (for example, "loading") for the corresponding on-board controller 300 based on the work status input by the operator, and transmits the generated work status data 401 to the on-board controller 300 via the WAN.

[0058] In S2101, the in-vehicle controller 300 checks whether or not work status data 401 has been input (transmitted) from the corresponding in-vehicle controller 300. If work status data 401 has been input, the in-vehicle controller 300 stores the work status data 401 in a table (S2102) and proceeds to S2203. On the other hand, if work status data 401 has not been input, the process proceeds directly to S2203.

[0059] In S2203, the in-vehicle controller 300 determines whether or not there is a transmission request for the work status data 401 from the image management server 200. If there is a transmission request, the in-vehicle controller 300 transmits the work status data 401 to the image management server 200 (S2204) and proceeds to S2205. On the other hand, if there is no transmission request, the in-vehicle controller 300 proceeds directly to S2205.

[0060] In S2205, the in-vehicle controller 300 determines whether or not there is a request from the image management server 200 to send the connection destination data 405. If there is a request, the in-vehicle controller 300 sends the connection destination data 405 to the image management server 200 (S2206) and proceeds to S2207. On the other hand, if there is no request, the in-vehicle controller 300 proceeds directly to S2207.

[0061] In S2207, the in-vehicle controller 300 determines whether or not the transmission data upper limit data 406 has been transmitted from the image management server 200. If the transmission data upper limit data 406 has been transmitted, the in-vehicle controller 300 stores the transmission data upper limit data 406 in a table in the storage device 312 (S2208) and proceeds to S2004. On the other hand, if the transmission data upper limit data 406 has not been transmitted, the in-vehicle controller 300 proceeds to S2204.

[0062] In S2004, the in-vehicle controller 300 sets the image quality of the video data from the multiple cameras 310a, 310b, and 310c mounted on the vehicle within the transmission data upper limit range in a table in the storage device 312. The image quality setting is performed using a known method. In addition to lowering the image quality of the video data from each camera 310 to match the transmission data upper limit value, the amount of transmitted data may be reduced by setting a camera 310 to suspend image capture depending on the work status. The image quality set for each camera 310 here is stored in the camera image quality data 402, 403, and 404.

[0063] In S2005, the in-vehicle controller 300 causes each camera 310 to capture video data with image quality according to the settings of the camera image quality data 402, 403, and 404.

[0064] In S2006, the on-board controller 300 transmits the video data captured by each camera 310 via the WAN to the corresponding management computer 100. As a result, the video data from the camera 310 is displayed on the monitor 120 connected to the management computer 100 with image quality that corresponds to the work status of the machine itself, and the operator can remotely control the work machine 1 by operating the operation device 150 while viewing the video data.

[0065] After transmitting the video data in S2006, the in-vehicle controller 300 returns to S2101 and repeats the subsequent processes in accordance with the flowchart of FIG.

[0066] (Actions and Effects) In the video transmission system configured as described above, the image management server 200 determines the priority of the transmission data volume of each on-board controller 300 based on the work status of each work machine 1 and the correspondence (priority table 502) between the work status and priority (priority score 550), and calculates an upper limit value (transmission data upper limit 406) for the transmission data volume of each on-board controller 300 based on the priority (priority score 550) of the transmission data volume of each on-board controller 300 so that the total data volume of video data from the cameras 310 transmitted from the multiple on-board controllers 300 to the network (WAN) is equal to or less than a predetermined value (e.g., 1000 Mbps). The calculated transmission data upper limit 406 is transmitted to the corresponding on-board controller 300, and each on-board controller 300 adjusts the image quality of the video data from the multiple cameras 310 so that the data volume is equal to or less than the data volume specified by the transmission data upper limit 406, and transmits the image quality-adjusted video data via the network to the management computer 100.

[0067] In this way, by determining the transmission data upper limit 406 for each on-board controller 300 based on the priority (priority score 550) set in advance for each work state of the work machine 1, it is possible to transmit to the management computer 100 video data of image quality suitable for the work state of each work machine 1. For example, in a "loading operation" in which a hydraulic excavator loads earth and sand into the bed of a dump truck, it is preferable that the operator of the management computer 100 be able to accurately grasp the position of the toe of the bucket, the position of the bed of the dump truck, and the load amount on the monitor 120. Therefore, if the priority (priority score 550) of the work state "loading" is set higher than that of other work states, as in this embodiment, the image quality of the video data from the camera 310 viewed by the operator through the monitor 120 during loading operation will be preferentially higher, thereby enabling loading operation by remote control to be carried out smoothly.

[0068] In this embodiment, the operator of the management computer 100 sets the work status (work status data 401) of each work machine 1 each time the work changes, but if the work status data 401 is left at its initial value (unspecified), the priority score 550 is set to the lowest value (1) (see S2001 in FIG. 11). For this reason, if the operator carries out a high-priority work (for example, loading work) while forgetting to set the work status, there is a high possibility that the image quality of the image on the monitor 120 will be low. In other words, according to this embodiment, the image quality of the monitor 120 can make the operator aware that he or she has forgotten to set the work status, and therefore it is possible to urge the operator to set the correct work status.

[0069] In this embodiment, a configuration is adopted in which the image management server 200 calculates the transmission data upper limit 406 when there are multiple vehicle controllers 300 connected to a certain base station 50, but it is also possible to adopt a configuration in which the image management server 200 calculates the transmission data upper limit 406 when there are multiple vehicle controllers 300 using the same communication route on the WAN.

[0070] Second Embodiment In the first embodiment, the operator of the management computer 100 manually set the working status of each hydraulic excavator 1 (see S2002 in FIG. 11), but the main feature of this embodiment is that the working status of each hydraulic excavator 1 is automatically set based on the position data of the work machine 1 and the map information table 3202 (see FIG. 13).

[0071] Fig. 12 is an overall configuration diagram of a video transmission system according to a second embodiment of the present invention. This system includes a plurality of dump trucks 2A, 2B as work machines for transporting earth and sand excavated by a hydraulic excavator 1, and each of the dump trucks 2A, 2B is equipped with an on-board controller 400A, 400B connected to a LAN. Each of the hydraulic excavators 1A, 1B, 1C, 1D and the dump trucks 2A, 2B is equipped with an antenna and a receiver (these will be referred to as a "positioning device 20") for detecting its own position using a positioning satellite system. The on-board controllers 300, 400 of each of the work machines 1, 2 store position data calculated by its own positioning device 20. This position data can be transmitted from the on-board controllers 300, 400 of each of the work machines 1, 2 to an image management server 200 via a LAN.

[0072] (Image management server 200 data) 13 is a diagram showing in table form part of the data stored in the storage device 212 of the image management server 200 according to the second embodiment of the present invention. As shown in this figure, the storage device 212 of the image management server 200 stores a location information table 3201 and a map information table 3202 in addition to the tables 501 and 502 shown in FIG. The position information table 3201 stores position data for each work machine (hydraulic excavators (vehicles) 1A, 1B, 1C, 1D and dump trucks (trucks) 2A, 2B) connected to the same LAN as the image management server 200. Each piece of position data is calculated by the positioning device 20, and defines the position by latitude and longitude. For example, the position of the vehicle body 1A is latitude Gx and longitude Gy.

[0073] The map information table 3202 stores position data for a plurality of areas obtained by dividing a work site into a plurality of areas. In this embodiment, each area is defined by a rectangle, and the position of each area is defined by the upper left vertex position (first vertex position) and the lower right vertex position (second vertex position) of the rectangle, as shown in Fig. 14. A work state of the hydraulic excavator 1 is assigned to each area in advance. The work state data 401 of each hydraulic excavator 1 (each on-board controller 300) in the vehicle body information table 501 (Fig. 6) is set to the work state assigned to the area in which the excavator is located.

[0074] However, it is difficult to distinguish between the two work states, "excavating" and "loading," based only on the map information table 3202 and the position data of the hydraulic excavator 1. Therefore, in this embodiment, the work state is first provisionally set to "excavating and loading," and then the distance between the hydraulic excavator 1 and the dump truck 2 is taken into consideration to finally determine whether it is "excavating" or "loading" (details will be described later using FIG. 16). Other work states (for example, leveling, cleaning, moving to garage, etc.) follow the settings in the map information table 3202.

[0075] (Processing of image management server 200) Fig. 15 is a flowchart of the processing executed by the image management server 200 of this embodiment. Here, steps S1011 and S1012, which are different from the flowchart in Fig. 9, will be explained, and other steps that are the same as those in Fig. 9 will be assigned the same reference numerals and will not be explained.

[0076] In S1011, the image management server 200 requests each on-board controller 300, 400 (work machines 1, 2) connected to the same LAN to send position data (position data of its own machine) obtained by the positioning device 20, and then stores the position data of each work machine 1, 2 sent from each on-board controller 300, 400 in the position information table 3201.

[0077] In S1012, the image management server 200 calculates the work status data 401 of the hydraulic excavator 1 under management based on the position data of each work machine 1, 2 acquired in S1011 and the data in the map information table 3202, and stores the calculated work status data 401 in the vehicle body information table 501 (FIG. 6). Specifically, the process of FIG. 16 is performed.

[0078] FIG. 16 is a flowchart showing the details of the process executed by the image management server 200 of this embodiment in S1012.

[0079] In S3501, the image management server 200 selects one hydraulic excavator 1 from the multiple hydraulic excavators 1 under management, extracts the position data of the selected hydraulic excavator 1 (selected excavator) 1 from the position information table 3201, identifies the area in which the selected excavator 1 is located from the map information table, and determines whether the work status assigned to that area is "digging and loading in progress." If the work status assigned to that area is "digging and loading in progress," the process proceeds to S3502, and if the work status is any other than the above (including when no work status is assigned), the process proceeds to S3505.

[0080] In S3502, the image management server 200 obtains the position data of the selected shovel 1 and the position data of the dump truck 2 closest to the selected shovel 1 from the position information table, and determines whether the distance between them is within a predetermined threshold (for example, 10 m). In the example of FIG. 14, the distance between the selected shovel 1 and dump truck 2 can be calculated from the "latitude Gx, longitude Gy" of shovel 1A and the "latitude Ux, longitude Uy" of dump truck 2A. If the distance between them is within the threshold (10 m), the selected shovel is considered to be loading the nearest dump truck 2, and "loading" is set as the work status data 401 (S3503). On the other hand, if the distance exceeds the threshold (10 m), the selected shovel is considered to be excavating, and "excavating" is set as the work status data 401 (S3504).

[0081] The threshold value used in S3502 is not limited to the example of 10 m, but can be changed as appropriate depending on the size of the hydraulic excavator or dump truck, the shape of the work site, etc.

[0082] In S3505, the image management server 200 determines whether or not a work status has been assigned to the area where the selected shovel 1 is located. If a work status has been assigned to the area, the image management server 200 sets the work status as the work status data 401 (S3506), and if no work status has been assigned, the image management server 200 sets predetermined work status data (for example, "unspecified") as the work status data 401 (S3607).

[0083] In S3508, the image management server 200 determines whether it has checked the areas to which all managed hydraulic excavators 1 belong, and if checking has been completed for all hydraulic excavators 1, it proceeds to S1003; if not, it returns to S3501.

[0084] (Processing of the in-vehicle controller 300) Next, we will explain the flow of processing executed by each on-board controller 300. Figure 17 shows a flowchart of processing executed by each on-board controller 300 according to this embodiment. Here, we will explain S2209 and S2210, which are processes different from the flowchart in Figure 11, and other processes that are the same as those in Figure 11 will be assigned the same reference numerals and will not be explained.

[0085] In S2209, the in-vehicle controller 300 determines whether or not there is a request to send position data from the image management server 200. If there is a request to send, the in-vehicle controller 300 sends the position data to the image management server 200 (S2210) and proceeds to S2205. On the other hand, if there is no request to send, the process proceeds directly to S2205.

[0086] Although no explanation using a flowchart will be provided, the on-board controller 400 of the dump truck 2 is configured to transmit position data in response to a request from the image management server 200, just like the on-board controller 300 of the hydraulic excavator 1.

[0087] (Actions and Effects) In this embodiment, the working status of each hydraulic excavator 1 is automatically set based on the position data of the work machine 1 and the map information table 3202 (see FIG. 13), which eliminates the need for the operator of the management computer 100 to set the working status, improving work efficiency. In addition, the fact that the working status will no longer be maintained at its initial value due to the operator forgetting to set the working status also contributes to improved work efficiency.

[0088] Furthermore, in this embodiment, the position data of not only the hydraulic excavator 1 but also the dump truck 2 is referenced, making it possible to accurately classify two work states, "loading" and "excavating," which are difficult to distinguish using only the position data of the hydraulic excavator 1.

[0089] However, the working status of the hydraulic excavator 1 may be set using only the position data of the hydraulic excavator 1 and the map information table, without referring to the position data of the dump truck 2. In this case, "loading" and "excavating" are classified using only the position data and the map information table.

[0090] (others) In each of the above embodiments, the case where video data from a camera 310 mounted on a hydraulic excavator 1 is transmitted has been described, but the present invention can also be applied to the case where video data from a camera mounted on a work machine other than the hydraulic excavator 1, such as a wheel loader, a crane, or a dump truck, is transmitted.

[0091] In each of the above embodiments, the case where the work machine 1 is remotely controlled by the management computer 100 has been described, but the present invention is also applicable to cases where the operating status of each work machine 1 is managed based on video data transmitted from each on-board controller 300.

[0092] In the second embodiment, the image management server 200 determines the working status of the hydraulic excavator 1 based on the position data of each of the work machines 1, 2 and the map information table 3202, but the map information table 3202 may be stored in the on-board controller 300, so that the working status of the hydraulic excavator 1 is determined on the on-board controller 300 side. In this case, for example, the algorithm of each on-board controller 300 may be configured to execute the processing for determining the working status based on the position data instead of the processing of S2002 in the flowchart of Fig. 11 .

[0093] The present invention is not limited to the above-described embodiments, and includes various modifications within the scope of the gist of the present invention. For example, the present invention is not limited to those having all of the configurations described in the above-described embodiments, and includes those in which some of the configurations are omitted. Furthermore, it is possible to add or replace some of the configurations of one embodiment with the configurations of another embodiment.

[0094] Furthermore, the configurations of the management computer 100, image management server 200, and on-board controllers 300 and 400 (hereinafter referred to as control devices 100, 200, 300, and 400), as well as their functions and execution processes, may be partially or entirely implemented in hardware (e.g., by designing logic for executing each function as an integrated circuit). Furthermore, the configurations of the control devices 100, 200, 300, and 400 may be implemented as programs (software) that are read and executed by an arithmetic processing device (e.g., a CPU) to implement the functions of the control devices 100, 200, 300, and 400. Information related to the programs may be stored in, for example, semiconductor memory (flash memory, SSD, etc.), magnetic storage devices (hard disk drives, etc.), and recording media (magnetic disks, optical disks, etc.).

[0095] In addition, in the above description of each embodiment, the control lines and information lines are those that are considered necessary for the description of the embodiment, but they do not necessarily represent all the control lines and information lines related to the product. In reality, it can be considered that almost all components are interconnected. [Explanation of symbols]

[0096] 1 (1A, 1B, 1C, 1D)... hydraulic excavator (work machine), 2 (2A, 2B) dump truck (work machine), 20... positioning device, 50 (50A, 50B)... base station, 100 (100A, 100B, 100C, 100D)... management computer, 112... storage device, 115... communication device, 120... monitor, 130... input device, 150... operation device, 200... image management server, 212... storage device, 215... communication device, 300 (300A, 300B, 300C, 300D)... on-board controller, 302... solenoid valve, 303... pressure sensor, 305... engine Gin, 306... rotation speed sensor, 310 (310a, 310b, 310c)... camera, 312... storage device, 315... communication device, 400 (400A, 400B)... vehicle controller, 401... work status data, 402... first camera image quality data, 403... second camera image quality data, 404... third camera image quality data, 405... connection destination data, 405... priority score data, 406... transmission data upper limit data, 501... vehicle body information table, 502... priority table, 550... priority score data, 3201... location information table, 3202... map information table

Claims

1. a plurality of on-board controllers that are mounted on a plurality of work machines each having a plurality of cameras and that transmit video data from the plurality of cameras to a network; an image management server connected to the network so as to be able to communicate with the plurality of vehicle-mounted controllers; a computer connected to the network so as to be able to communicate with the plurality of on-board controllers, receiving video data from the plurality of cameras transmitted from the plurality of on-board controllers and displaying the video data on a monitor, The image management server determining the priorities of the plurality of on-board controllers based on the work states of the plurality of work machines and a correspondence relationship between the work states of the work machines and priorities that is defined by setting a priority in advance for each work state of the work machines; calculating an upper limit value for the amount of data transmitted from each of the in-vehicle controllers based on the priority of the in-vehicle controllers so that the total amount of video data from the plurality of cameras transmitted from the plurality of in-vehicle controllers to the computer is equal to or less than a predetermined value; The calculated upper limit of the amount of data to be transmitted is transmitted to each vehicle controller. each of the plurality of in-vehicle controllers adjusts the image quality of the video data from the plurality of cameras so that the image quality is within the upper limit value of the amount of data transmitted from the image management server, and transmits the image data after the image quality adjustment to the computer; the plurality of work machines are a plurality of hydraulic excavators, the work status of the work machine includes "loading" indicating that the hydraulic excavator is performing loading work onto the bed of a dump truck, and "excavating" indicating that the hydraulic excavator is performing excavation work, The priority defined by the correspondence relationship between the work state of the work machine and the priority is set highest for "loading," The image management server calculates the upper limit value of the transmission data amount so that the upper limit value of the transmission data amount is larger for an in-vehicle controller with a relatively higher priority. A video transmission system for a work machine.

2. 2. The video transmission system for a work machine according to claim 1, The plurality of on-board controllers store position data of their own vehicle calculated by a positioning device mounted on their own vehicle, The image management server setting the work status of the plurality of work machines based on the position data of the own machine transmitted from the plurality of on-board controllers and a map information table that defines the work statuses performed by the work machines in a plurality of areas set in the work site where the plurality of work machines are operating; The priority of the plurality of on-board controllers is determined based on the set work states of the plurality of work machines and the correspondence between the work states of the work machines and the priorities. A video transmission system for a work machine.

3. 3. The video transmission system for a work machine according to claim 2, The image management server when the working state of the hydraulic excavator, which has been set based on the position data and the map information table, is "digging and loading" and when the distance between the hydraulic excavator and the dump truck closest to the hydraulic excavator is equal to or less than a predetermined threshold, the working state of the hydraulic excavator is set to "loading," When the working state of the hydraulic excavator set based on the position data and the map information table is "digging and loading" and the distance between the hydraulic excavator and the nearest dump truck exceeds the threshold value, the working state of the hydraulic excavator is set to "digging." A video transmission system for a work machine.

4. 2. The video transmission system for a work machine according to claim 1, In the communication path from the plurality of vehicle controllers to the computer, the base station through which the image data of the plurality of cameras transmitted from the plurality of vehicle controllers first passes is the same. A video transmission system for a work machine.

5. 2. The video transmission system for a work machine according to claim 1, an operation device is connected to the computer and outputs an operation signal via the network to a work machine on which the image data from the plurality of cameras is displayed on the monitor; The operation device is operated by an operator who watches the video data of the plurality of cameras displayed on the monitor. A video transmission system for a work machine.

6. In the video transmission system for a work machine of claim 1, The working state of the work machine is set by an operator, the work state of the work machine includes “not specified,” which indicates that the work state has not been set by the operator after the on-board controller is started, The priority defined in the correspondence relationship between the work state of the work machine and the priority is set to "not specified" as the lowest. A video transmission system for a work machine.

Images