Display control system and display method

Abstract

Have the operator of the work machine confirm the positional relationship between the work machine they are operating and other work machines. [Solution] The processor identifies the positional relationship between the first work machine and the second work machine located near the first work machine. The processor causes the display unit to display a display screen that includes a positional relationship diagram in which model images of the first work machine and the second work machine are arranged according to their positional relationship, and status information of the second work machine.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a display control system and a display method.

BACKGROUND ART

[0002] Patent Document 1 discloses a technique for displaying guidance information representing the positional relationship between a hydraulic excavator and a dump truck on a monitor of the hydraulic excavator.

PRIOR ART DOCUMENTS

PATENT DOCUMENTS

[0003]

PATENT DOCUMENT 1

SUMMARY OF THE INVENTION

PROBLEM TO BE SOLVED BY THE INVENTION

[0004] The appropriate positional relationship between a hydraulic excavator for loading a load onto a dump truck and the dump truck may differ depending on the construction environment, construction conditions, the preference of the operator who operates the hydraulic excavator, and the like. Therefore, it is required to enable the operator of the hydraulic excavator to check the positional relationship between the hydraulic excavator and the dump truck. An object of the present disclosure is to provide a display control system and a display method that allow an operator of a work machine to check the positional relationship between the target work machine being operated and another work machine.

MEANS FOR SOLVING THE PROBLEM

[0005] According to one aspect of the present invention, a display control system that displays a screen presented to an operator who operates a first work machine includes a display unit and a processor, identifies a positional relationship between the first work machine and a second work machine located in the vicinity of the first work machine, and causes the display unit to display a display screen including a positional relationship diagram in which a model image of the first work machine and a model image of the second work machine are arranged according to the positional relationship, and state information of the second work machine. [Effects of the Invention]

[0006] According to the above embodiment, the display control system can allow the operator of a work machine to confirm the positional relationship between the work machine being operated and other work machines. [Brief explanation of the drawing]

[0007] [Figure 1] This diagram shows the relationship between the excavating machine and the transport vehicle according to the first embodiment. [Figure 2] This is a schematic diagram showing the configuration of the excavation machine according to the first embodiment. [Figure 3] This is a schematic block diagram showing the configuration of the control device according to the first embodiment. [Figure 4] This figure shows an example of a display screen according to the first embodiment. [Figure 5] This is a flowchart (Part 1) showing the display method of the display screen according to the first embodiment. [Figure 6] This is a flowchart (Part 2) showing the display method of the display screen according to the first embodiment. [Figure 7] This figure shows an example of a display screen according to another embodiment. [Modes for carrying out the invention]

[0008] <First Embodiment> Figure 1 shows the relationship between the excavating machine and the transport vehicle according to the first embodiment. The excavating machine 100 operates at the construction site, excavating soil and other materials, and loading them onto the transport vehicle 200 as cargo. In the first embodiment, the excavating machine 100 is a hydraulic excavator. The transport vehicle 200 loads the soil and other materials excavated by the excavating machine 100 onto its cargo bed and transports them. In the first embodiment, the transport vehicle 200 is a dump truck. The excavating machine 100 and the transport vehicle 200 are examples of work machinery.

[0009] The transport vehicle 200 is equipped with a positioning device 210 and a communication device 220. The positioning device 210 calculates the position of the transport vehicle 200 in the site coordinate system and the direction the transport vehicle 200 is facing. The positioning device 210 is equipped with two antennas that receive positioning signals from satellites that constitute the GNSS. The two antennas are installed at different positions on the transport vehicle 200. Based on the positioning signals received by at least one of the two antennas, the positioning device 210 detects the position of the transport vehicle 200 in the site coordinate system. Using the positioning signals received by each of the two antennas, the positioning device 210 detects the direction the transport vehicle 200 is facing in the site coordinate system. The communication device 220 transmits information indicating the position and direction measured by the positioning device 210 to the communication device 160 of the excavation machine 100.

[0010] 《Configuration of Excavation Machine 100》 Figure 2 is a schematic diagram showing the configuration of the excavation machine according to the first embodiment. The excavator 100 according to the first embodiment has a function to show the operator the positional relationship between the excavator 100 and the transport vehicle 200.

[0011] The excavation machine 100 includes a traveling body 110, a rotating body 120, a working machine 130, a driver's cab 140, and a control device 150. The traveling body 110 supports the excavator 100 so that it can move. The traveling body 110 is an example of a support for the excavator 100. The traveling body 110 is, for example, a pair of endless tracks on the left and right. The slewing body 120 is supported on the traveling body 110 so that it can slewing around a pivot point. The work implement 130 is hydraulically driven. The work implement 130 is supported on the front of the slewing body 120 so that it can be driven vertically. The cab 140 is a space for the operator to sit in and operate the excavator 100. The cab 140 is located on the front of the slewing body 120. The control device 150 controls the traveling body 110, the slewing body 120, and the work implement 130 based on the operator's operations. The control device 150 is located, for example, inside the cab 140.

[0012] The rotating body 120 includes a positioning device 121 and a plurality of imaging devices 122.

[0013] The positioning device 121 calculates the position of the rotating body 120 in the field coordinate system and the direction in which the rotating body 120 is facing. The positioning device 121 is equipped with two antennas that receive positioning signals from satellites that constitute the GNSS. The two antennas are installed at different locations on the rotating body 120. For example, the two antennas are installed on the counterweight portion of the rotating body 120. Based on the positioning signals received by at least one of the two antennas, the positioning device 121 detects the position of a representative point of the rotating body 120 in the field coordinate system. The representative point may be, for example, a point on the rotational axis of the rotating body 120. Using the positioning signals received by each of the two antennas, the positioning device 121 detects the direction in which the rotating body 120 is facing in the field coordinate system.

[0014] The multiple imaging devices 122 each capture images of the front, rear, left, and right sides of the excavation machine 100. The imaging ranges of adjacent imaging devices 122 partially overlap, and the combined imaging range of the multiple imaging devices 122 covers the entire circumference of the excavation machine 100. In other embodiments, the excavation machine 100 may be equipped with an omnidirectional camera instead of the multiple imaging devices 122. In other embodiments, the imaging range of the imaging devices 122 does not necessarily have to cover the entire circumference.

[0015] The work machine 130 comprises a boom 131, an arm 132, and a bucket 133. The base end of the boom 131 is attached to the slewing body 120 via a boom pin. The arm 132 connects the boom 131 and the bucket 133. The base end of the arm 132 is attached to the tip of the boom 131 via an arm pin. The bucket 133 comprises a cutting edge for excavating soil and other materials and a storage section for collecting the excavated soil and other materials. The base end of the bucket 133 is attached to the tip of the arm 132 via a bucket pin.

[0016] The cab 140 is a place where the operator of the excavating machine 100 boards to perform operations and steering. The cab 140 is, for example, arranged on the left portion of the front end of the revolving structure 120. The cab 140 of the excavating machine 100 is provided with a monitor 141 and an operation device 142 for operating the excavating machine 100. The monitor 141 presents information for assisting the operator of the excavating machine 100 in loading loads onto the transport vehicle 200. The operation device 142 is operated by the operator, and outputs an operation signal in response to the operator's operation.

[0017] <<Configuration of Control Device 150>> The control device 150 generates information for allowing the operator of the excavating machine 100 to confirm the positional relationship between the excavating machine 100 and the transport vehicle 200, based on the position and orientation data measured by the positioning device 121 and the position and orientation data of the transport vehicle 200 received from the transport vehicle 200, and causes the monitor 141 to display the information.

[0018] Figure 3 is a schematic block diagram showing the configuration of the control device according to the first embodiment. The control device 150 is a computer including a processor 510, a main memory 530, a storage 550, and an interface 570.

[0019] The storage 550 is a non-transitory tangible storage medium. Examples of the storage 550 include magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and the like. The storage 550 may be an internal medium directly connected to the bus of the control device 150, or may be an external medium connected to the control device 150 via the interface 570 or a communication line. The storage 550 stores a program for controlling the excavating machine 100.

[0020] The program may be for implementing a part of the functions to be performed by the control device 150. For example, the program may perform functions in combination with other programs already stored in the storage 550, or in combination with other programs implemented in other devices. In other embodiments, the control device 150 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to, or instead of, the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions implemented by the processor may be implemented by the integrated circuit.

[0021] The storage 550 stores setting information for the target position of the transport vehicle 200 during loading, which is set by the operator. The setting information for the target position includes, for example, the target distance from the pivot center of the excavator 100 to the transport vehicle 200, the orientation of the transport vehicle 200 relative to the excavator 100 (longitudinal or lateral), and the tolerances for the longitudinal and lateral directions. When the transport vehicle 200 is longitudinal relative to the excavator 100, it means that the directions that the excavator 100 and the transport vehicle 200 are facing are approximately the same. When the transport vehicle 200 is lateral relative to the excavator 100, it means that the direction that the excavator 100 is facing and the direction that the transport vehicle 200 is facing intersect at approximately 90 degrees.

[0022] The processor 510 is connected to the positioning device 121, the imaging device 122, and the communication device 160 via the interface 570. The communication device 160 is connected to the transport vehicle 200 via wireless communication.

[0023] The processor 510, by executing a program, includes a data receiving unit 511, a data acquisition unit 512, a position relationship identification unit 513, a person detection unit 514, and a display control unit 515.

[0024] The data receiving unit 511 receives position and orientation measurement data from the transport vehicle 200. The data acquisition unit 512 acquires position and direction measurement data from the positioning device 121 and acquires imaging data from multiple imaging devices 122.

[0025] The position relationship determination unit 513 determines the positional relationship between the excavator 100 and the transport vehicle 200 based on the position and orientation measurement data of the transport vehicle 200 received by the data receiving unit 511 and the position and orientation measurement data of the excavator 100 acquired by the data acquisition unit 512. Specifically, the position relationship determination unit 513 calculates the distance from the rotation center of the excavator 100 to the reference point of the transport vehicle 200 (the rear or side of the transport vehicle 200), the direction in which the transport vehicle 200 is located relative to the rotation center of the excavator 100, and the height difference between the excavator 100 and the transport vehicle 200. The reference point of the transport vehicle 200 is the center point of the rear of the transport vehicle 200 when the orientation of the transport vehicle 200 relative to the excavator 100 set as target position setting information is vertical, and is the center point of the left or right side of the transport vehicle 200 when the orientation of the transport vehicle 200 relative to the excavator 100 set as target position setting information is horizontal.

[0026] The person detection unit 514 performs person detection processing on the portion of the captured image acquired by the data acquisition unit 512 that includes a predetermined range containing the target location. The person detection processing may be performed by pattern matching using dictionary data, or by using a pre-trained machine learning model such as a CNN (Convolutional Neural Network).

[0027] The display control unit 515 uses the calculation results from the position relationship identification unit 513 and the person detection unit 514 to generate a display screen G for the operator to confirm the positional relationship between the excavation machine 100 and the transport vehicle 200, and displays it on the monitor 141.

[0028] [Displayed Image] Figure 4 shows an example of a display screen according to the first embodiment. The display screen G includes a top view G1, a positional relationship display G2, a positional relationship message G3, and an alert area G4.

[0029] The top view G1 includes an excavator image G11, which is a model image of the excavator 100; a transport vehicle image G12, which is a model image of the transport vehicle 200; a distance display image G13, which shows the distance between the excavator 100 and the transport vehicle 200; a target position image G14, which shows the target position of the transport vehicle 200; a direction image G15, which shows the deviation of the direction the transport vehicle 200 is facing relative to the direction the excavator 100 is facing; and a marker image G16, which represents a detected person. Note that the top view G1 according to the first embodiment is an example of a diagram showing the positional relationship between the excavator 100 and the transport vehicle 200. For example, in other embodiments, instead of the top view G1, a side view showing the positional relationship between the excavator 100 and the transport vehicle 200 from the side, or an overhead view rendered from a camera obliquely above the three-dimensional models of the excavator 100 and the transport vehicle 200 may be displayed.

[0030] The vertical direction in the top view G1 corresponds to the distance between the excavator 100 and the transport vehicle 200. In other words, the direction of the straight line connecting the pivot center of the excavator image G11 and the reference point of the transport vehicle image G12, and the direction in which the distance display image G13 extends, coincide with the vertical direction in the top view G1. The excavator image G11 is positioned at a fixed location below the top view G1. The excavator image G11 may display only a portion of the excavator 100, as shown in Figure 4. The excavator image G11 is rotated by the angle formed by the straight line connecting the pivot center of the excavator 100 and the reference point of the transport vehicle 200, and the straight line indicating the direction in which the excavator 100 is facing.

[0031] The transport vehicle image G12 is positioned such that its reference point is located above the excavator image G11 in the top view G1 by a length corresponding to the distance between the excavator 100 and the transport vehicle 200. The transport vehicle image G12 is rotated by the angle formed by the straight line connecting the pivot center of the excavator 100 and the transport vehicle 200, and the straight line indicating the direction the transport vehicle 200 is facing.

[0032] Distance indicator image G13 is an arrow with a length corresponding to the distance between the excavator 100 and the transport vehicle 200. Distance indicator image G13 is positioned between the excavator image G11 and the transport vehicle image G12. Distance indicator image G13 is positioned to face upwards in the top view G1. The target position image G14 is positioned in the top view G1 at a distance from the transport vehicle image G12 that corresponds to the distance indicated by the target position setting information. In other words, the target position of the transport vehicle 200 is at a distance indicated by the setting information from the rotation center of the excavator 100 in the direction in which the reference point of the transport vehicle 200 is located. The target position image G14 is drawn with a rectangular frame. The width of the target position image G14 is the width of the transport vehicle image G12 plus the tolerance in the left-right direction set in the setting information. The length of the target position image G14 is the length of the transport vehicle image G12 plus the tolerance in the front-back direction set in the setting information. If the orientation of the transport vehicle 200 relative to the excavator 100, as set in the target position setting information, is vertical, the longitudinal direction of the target position image G14 coincides with the vertical direction of the top view G1. If the orientation of the transport vehicle 200 relative to the excavator 100, which is set as the target position setting information, is sideways, then the longitudinal direction of the target position image G14 coincides with the left-right direction of the top view G1.

[0033] Directional image G15 is drawn with an arc whose central angle is the angle of the direction in which the transport vehicle 200 faces relative to the distance direction between the excavator 100 and the transport vehicle 200. Directional image G15 is positioned so that the center of the arc coincides with the reference point of the transport vehicle image G12. The marker image G16 is placed at the position corresponding to the location where a person is detected in the top view G1. If no person is detected by the person detection unit 514, the marker image G16 is not placed.

[0034] The positional relationship display G2 includes distance information G21, elevation difference information G22, and angle information G23. Distance information G21 represents the distance from the pivot center of the excavator 100 to the reference point of the transport vehicle 200. Elevation difference information G22 represents the difference between the height of the excavator 100 and the height of the transport vehicle 200. Angle information G23 represents the angle in the direction the transport vehicle 200 faces relative to the distance direction between the excavator 100 and the transport vehicle 200. In addition, the positional relationship display G2 may also include information on the size of the transport vehicle 200. The size of the transport vehicle 200 may include, for example, the width of the transport vehicle and the height of the side panels of the cargo bed. The size information of the transport vehicle 200 may be received from the transport vehicle 200 or estimated from the image captured by the imaging device 122.

[0035] The positional relationship message G3 displays a message regarding the relationship between the target position and the position of the transport vehicle 200. When the reference point of transport vehicle 200 is located closer to the target position than the allowable error, positional message G3 will display a message such as "The dump truck is too close." When the difference between the reference point and the target position of the transport vehicle 200 does not exceed the reference point, or when the reference point is located further away from the target position and the transport vehicle 200 is approaching, the positional message G3 will display text such as "A dump truck is approaching." When the difference between the reference point and the target position of the transport vehicle 200 does not exceed the reference point, and the transport vehicle 200 is stopped, the position relationship message G3 will display a message such as "The dump truck is in the correct position." When the reference point of the transport vehicle 200 is located farther than the target position, and the transport vehicle 200 is stopped, the positional message G3 will display a message such as "The dump truck is too far away."

[0036] Alert area G4 will notify you if a person is detected near the target location. Specifically, if no person is detected in the area corresponding to top view G1, alert area G4 will display a message such as "There is no danger in the vicinity." If a person is detected in the area corresponding to top view G1, alert area G4 will display a message such as "There is a person nearby!" If a person is detected in the area corresponding to top view G1 and then no longer detected, alert area G4 will display a message such as "A person was nearby. Please be careful." for a certain period of time.

[0037] 《Display method》 While the excavation machine 100 is in operation, the control device 150 provides display control to assist the operator in loading onto the transport vehicle 200. Figure 5 is a flowchart (Part 1) showing the display method of the display screen according to the first embodiment. Figure 6 is a flowchart (Part 2) showing the display method of the display screen according to the first embodiment.

[0038] The data acquisition unit 512 acquires position and orientation measurement data from the positioning device 121 and image data from the multiple imaging devices 122 (step S1). The position relationship identification unit 513 identifies the position and orientation of the transport vehicle 200 based on the image data acquired in step S2, with respect to the front of the rotating body 120 of the excavating machine 100 as the reference (step S2). For example, the position relationship identification unit 513 may use a machine learning model to extract the transport vehicle 200 that appears in the image and identify the position and orientation of the transport vehicle 200 from the position and orientation of the transport vehicle 200 in the image. For example, the position relationship identification unit 513 may use a machine learning model that has been trained to take an image as input and output position and orientation. A training dataset for such a machine learning model can be obtained by the following procedure. First, models of the excavating machine 100 and transport vehicle 200 are randomly placed in a three-dimensional virtual space. An image of the transport vehicle 200 is rendered from a virtual camera fixed in front of the rotating body 120 of the excavator 100 and used as an input sample. The position and orientation of the transport vehicle 200 relative to the excavator 100 in the three-dimensional virtual space at this time are used as an output sample. Alternatively, a training dataset may be created using a simulator. For example, the simulator may output an image of the transport vehicle 200 rendered from a virtual camera fixed in front of the rotating body 120 of the excavator 100, when models of the excavator 100 and the transport vehicle 200 are randomly placed in the three-dimensional virtual space, as an input sample for the training dataset. The simulator may also calculate the position and orientation of the transport vehicle 200 relative to the excavator 100 in the three-dimensional virtual space at this time, as well as the region where the object is visible in the virtual camera image (segmentation) and features (keypoints, bounding boxes, etc.), and output these as output samples for the training dataset.

[0039] The position relationship identification unit 513 reads the target position setting information and identifies the distance and tolerance of the target position, as well as the reference point of the transport vehicle 200 (step S3). Based on the position of the transport vehicle 200 relative to the excavator 100 identified in step S2 and the reference point of the transport vehicle 200 identified in step S3, the position relationship identification unit 513 identifies the distance from the pivot center of the excavator 100 to the reference point of the transport vehicle 200, the altitude difference between the excavator 100 and the transport vehicle 200, and the direction of the half-line (distance direction) from the pivot center to the reference point (step S4).

[0040] The positional relationship determination unit 513 determines the angle of the transport vehicle 200 with respect to the distance direction based on the direction the transport vehicle 200 is facing, determined in step S2, and the distance direction, determined in step S4 (step S5).

[0041] The human detection unit 514 determines the range (the range shown in the top view G1) of the image data acquired in step S2 to be subject to human detection processing (step S6) based on the distance direction identified in step S4. The range displayed in the top view G1 is determined by a predetermined width and length based on the distance direction determined in step S4. The human detection unit 514 performs human detection processing on the range determined in step S6 from the image data acquired in step S2 (step S7).

[0042] The display control unit 515 creates the top view G1 by aligning the vertical direction of the top view G1 with the distance direction identified in step S4, and arranging the excavator image G11, the transport vehicle image G12, the distance display image G13, the target position image G14, and the direction image G15 according to the position and orientation of the excavator 100 and the transport vehicle 200 (step S8).

[0043] The display control unit 515 determines whether a person has been detected in the area to be displayed on the top view G1 by the person detection process in step S6 (step S9). If a person is detected (step S9: YES), the display control unit 515 places a marker image G16 at the position corresponding to the location where the person was detected in the top view G1 (step S10), and places a message indicating that a person is present in the alert area G4 (step S11). If no person is detected (step S9: NO), the display control unit 515 does not place a marker image G16 and places a message indicating that there is no danger in the alert area G4 (step S12).

[0044] The display control unit 515 places an image including distance information G21 indicating the distance identified in step S4, elevation difference information G22 indicating the elevation difference identified in step S4, and angle information G23 indicating the angle identified in step S5 on the positional relationship display G2 (step S13).

[0045] The display control unit 515 determines whether the reference point of the transport vehicle 200 is located closer to the target position than the allowable error, based on the distance and tolerance of the target position identified in step S3 and the distance identified in step S4 (step S14). If the reference point of the transport vehicle 200 is located closer to the target position than the allowable error (step S14: YES), the display control unit 515 places a message indicating that the transport vehicle 200 is too close in the position relationship message G3 (step S15). At this time, the control device 150 may automatically sound a horn to indicate that it is too close to the transport vehicle 200. For example, the control device 150 may continuously sound a long-duration horn intermittently.

[0046] If the difference between the reference point and the target position of the transport vehicle 200 does not exceed the reference point, or if the reference point is located further away from the target position (step S14: NO), the display control unit 515 determines whether or not the transport vehicle 200 is approaching based on the measurement data received in step S1 (step S16). If the transport vehicle 200 is approaching (step S16: YES), the display control unit 515 places a message indicating that the transport vehicle 200 is approaching in the position relationship message G3 (step S17).

[0047] On the other hand, if the transport vehicle 200 is stopped (step S16: NO), the display control unit 515 determines whether the difference between the reference point and the target position of the transport vehicle 200 is within the allowable error (step S18). If the difference between the reference point and the target position of the transport vehicle 200 is within the allowable error (step S18: YES), the display control unit 515 places a message in the position relationship message G3 indicating that the transport vehicle 200 is in the appropriate position (step S19). At this time, the control device 150 may automatically sound a horn to inform the transport vehicle 200 that it is in the appropriate position. For example, the control device 150 sounds a short horn once. On the other hand, if the difference between the reference point and the target position of the transport vehicle 200 exceeds the allowable error (step S18: NO), the display control unit 515 places a message in the position relationship message G3 indicating that the transport vehicle 200 is too far away (step S20).

[0048] The display control unit 515 generates a display screen G including a top view G1, a positional relationship display G2, a positional relationship message G3, and an alert area G4, and outputs a signal to the monitor 141 to display the display screen G (step S21).

[0049] "effect" As described above, the control device 150 according to the first embodiment displays a display screen G on the monitor 141, which includes a top view G1 (positional relationship diagram) in which an image of the excavating machine G11 and an image of the transport vehicle G12 are arranged according to the positional relationship between the excavating machine 100 and the transport vehicle 200, and a positional relationship display G2 (status information) showing the status of the transport vehicle 200. This allows the control device 150 to allow the operator of the excavating machine 100 to confirm the positional relationship between the excavating machine 100 and the transport vehicle 200.

[0050] <Other Embodiments> Although one embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to that described above, and various design changes are possible. In other embodiments, the order of the above-described processes may be changed as appropriate. Also, some processes may be executed in parallel. The control device 150 according to the above embodiment may be composed of a single computer, or the configuration of the control device 150 may be divided among multiple computers, and the multiple computers may cooperate with each other to function as the control device 150. In this case, some of the computers constituting the control device 150 may be mounted inside the excavation machine 100, while the other computers may be provided outside the excavation machine 100.

[0051] The work machines according to the above-described embodiment are a hydraulic excavator (excavation machine 100) and a dump truck (transport vehicle 200), but are not limited to these. For example, the work machines according to other embodiments may be other work machines such as wheel loaders, bulldozers, and cranes.

[0052] The work machine (excavator 100) according to the above embodiment is operated by an operator who is seated in the operator's cab 140, but is not limited to this. For example, work machines according to other embodiments may be operated by remote control. In this case, the monitor 141 and the operating device 142 are located remotely from the work machine. In this case as well, the monitor 141 is located near the operating device 142, and the display screen G is presented to the operator during operation. In remote operation, since the operator views images captured by a camera installed on the work machine on a display, it is more difficult to recognize the positional relationship (especially in the depth direction) between the work machine being operated and other work machines compared to when the operator is seated on the work machine. Therefore, by displaying the positional relationship between the work machine being operated and other work machines on the monitor 141, the difficulty in spatial awareness in remote operation can be mitigated.

[0053] The control device 150 according to the above-described embodiment displays a message in the position relationship message G3 that corresponds to the distance between the excavating machine 100 and the transport vehicle 200. In contrast, the control device 150 according to the other embodiment may also display a message regarding the lateral displacement. For example, in step S23, if the distance between the target position and the reference point is within the allowable error, the control device 150 according to the other embodiment may place a message in the position relationship message G3 indicating that the transport vehicle 200 is laterally displaced if the lateral position of the transport vehicle 200 exceeds the allowable error in the lateral direction. For example, the control device 150 may place a message in the position relationship message G3 indicating that the position is appropriate when the transport vehicle image G12 is inside the target position image G14, and place a message in the position relationship message G3 indicating that the position is inappropriate when the transport vehicle image G12 is outside the target position image G14.

[0054] The control device 150 according to the above embodiment determines the position and orientation of the transport vehicle 200 by receiving position and orientation measurement data from the transport vehicle 200, but is not limited to this. For example, the control device 150 according to another embodiment may determine the relative positional relationship between the excavation machine 100 and the transport vehicle 200 based on imaging data acquired from the imaging device 122. In this case, the control device 150 can determine the positional relationship between the excavation machine 100 and the transport vehicle 200 without using measurement data from the positioning device 121.

[0055] The control device 150 according to the above embodiment identifies the relative positional relationship between the excavator 100 and the transport vehicle 200 from the image captured by the imaging device 122, but is not limited to this. For example, the control device 150 according to another embodiment may identify the relative positional relationship between the excavator 100 and the transport vehicle 200 from distance images obtained by a LiDAR or stereo camera. Alternatively, the control device 150 according to another embodiment may identify the relative positional relationship between the excavator 100 and the transport vehicle 200 from the position and orientation of the transport vehicle 200 received by the data receiving unit 511 from the transport vehicle 200, and the position and orientation of the excavator 100. Furthermore, the control device 150 may identify the positional relationship by combining these multiple means.

[0056] The control device 150 according to the above embodiment sets the target position and generates the top view G1 using the distance direction, which is the direction of the half-line from the pivot center of the excavating machine 100 toward the reference point, as the reference direction, but is not limited to this. For example, the control device 150 according to another embodiment may use a direction specified by the operator as the reference direction. The operator can specify the reference direction, for example, by operating the monitor 141. In addition, the control device 150 according to another embodiment may use the front direction of the slewing body 120 or the front direction of the traveling body 110 as the reference direction.

[0057] The control device 150 according to the above embodiment displays a target position image G14 and a direction image G15 on the display screen G shown in Figure 4 as images representing the positional relationship between the excavator 100 and the transport vehicle 200. The control device 150 according to other embodiments may further display an image representing the widthwise displacement when the orientations of the excavator 100 and the transport vehicle 200 coincide. Figure 7 is a diagram showing an example of the display screen G according to another embodiment. In addition to the image of the first embodiment, the display screen G shown in Figure 7 further includes a front-facing direction image G17 and a right-angle direction image G18. The front-facing direction image G17 is a straight line that passes through the rotation center of the excavator 100 and extends in a direction that coincides with the orientation of the transport vehicle 200. The displacement between the front-facing direction image G17 and the transport vehicle image G12 represents the widthwise displacement when the orientations of the excavator 100 and the transport vehicle 200 coincide. [Explanation of Symbols]

[0058] 100…Excavator 110…Traveling body 120…Slewing body 121…Positioning device 122…Imaging device 130…Working equipment 131…Boom 132…Arm 133…Bucket 140…Operator's cab 141…Monitor 142…Operating device 150…Control device 150…Control device 160…Communication device 200…Transport vehicle 210…Positioning device 220…Communication device 510…Processor 511…Data receiving unit 512…Data acquisition unit 513…Position relationship identification unit 514…Human detection unit 515…Display control unit 530…Main memory 550…Storage 570…Interface G…Display screen G1…Top view G11…Excavator image G12…Transport vehicle image G13…Distance display image G14…Target position image G15…Direction image G16…Marker image G2…Position relationship display G21…Distance information G22…Elevation difference information G23…Angle information G3…Position relationship message G4…Alert area

Claims

1. A display control system that displays a screen for the operator of the first work machine, It comprises a display unit and a processor, The aforementioned processor, The positional relationship between the first working machine and the second working machine located in the vicinity of the first working machine is determined. The display unit displays a display screen that includes a positional relationship diagram in which model images of the first work machine and the second work machine are arranged according to the positional relationship, and status information of the second work machine. Display control system.

2. The status information of the second working machine includes positional relationship information that represents the positional relationship between the first working machine and the second working machine. The display control system according to claim 1.

3. The positional relationship information includes the following values: the second work machine is approaching the first work machine; the positional relationship between the first work machine and the second work machine is appropriate; and the distance between the first work machine and the second work machine is too close. The display control system according to claim 2.

4. The display screen includes a target image representing the target position of the second working machine relative to the first working machine. The first working machine comprises a support portion that supports the first working machine, a slewing body rotatably supported by the support portion, and a working machine provided on the slewing body. The aforementioned target image is positioned on a straight line connecting the pivot center of the rotating body and the second working machine. The display control system according to claim 1.

5. The aforementioned status information includes information about people who are located within the range represented by the positional relationship diagram. The information about the person is a warning indicating the presence of the person in the vicinity of the first work machine. The display control system according to claim 4.

6. The display screen includes numerical values ​​indicating the positional relationship between the first working machine and the second working machine. The display control system according to claim 1.

7. The numerical values ​​indicating the positional relationship include the distance between the first work machine and the second work machine, the difference in elevation between the first work machine and the second work machine, or the angle of the second work machine with respect to the straight line connecting the first and second work machines. The display control system according to claim 6.

8. The aforementioned state information includes information representing the widthwise displacement when the orientations of the first working machine and the second working machine are aligned. The display control system according to claim 1.

9. The status information includes a numerical value indicating the size of the second work machine. The display control system according to claim 1.

10. The aforementioned first working machine is a shovel, The second piece of machinery mentioned above is a dump truck. The display control system according to claim 1.

11. The display unit is provided near the operating device for operating the first work machine. The display control system according to claim 1.

12. A display method for showing a screen to an operator who is operating the first work machine, Computers The steps include determining the positional relationship between the first working machine and a second working machine located in the vicinity of the first working machine, The step of displaying a display screen on a display device that includes a positional relationship diagram in which model images of the first work machine and the second work machine are arranged according to the positional relationship, and status information of the second work machine. A method for displaying the execution.

13. The status information of the second working machine includes positional relationship information that represents the positional relationship between the first working machine and the second working machine. The display method according to claim 12.

14. The positional relationship information includes the following values: the second work machine is approaching the first work machine; the positional relationship between the first work machine and the second work machine is appropriate; and the distance between the first work machine and the second work machine is too close. The display method according to claim 13.

15. The display screen includes a target image representing the target position of the second working machine relative to the first working machine. The first working machine comprises a support portion that supports the first working machine, a slewing body rotatably supported by the support portion, and a working machine provided on the slewing body. The aforementioned target image is positioned on a straight line connecting the pivot center of the rotating body and the second working machine. The display method according to claim 12.

16. The aforementioned status information includes information about people who are located within the range represented by the positional relationship diagram. The information about the person is a warning indicating the presence of the person in the vicinity of the first work machine. The display method according to claim 15.

17. The display screen includes numerical values ​​indicating the positional relationship between the first working machine and the second working machine. The display method according to claim 12.

18. The numerical values ​​indicating the positional relationship include the distance between the first and second working machines, the difference in elevation between the first and second working machines, and the angle of the second working machine with respect to the straight line connecting the first and second working machines. The display method according to claim 17.

19. The aforementioned first working machine is a shovel, The second piece of machinery mentioned above is a dump truck. The display method according to claim 12.

20. The display device is provided near the operating device for operating the first work machine. The display method according to claim 12.

Images