Display control system and display control method

Abstract

To effectively analyze operations of a work machine.SOLUTION: A display control system comprises: an acquisition unit that acquires chronically-mapped log information of a work machine; an image generation unit that sequentially generates operation images of the operations of the work machine by sequentially applying operation information based on the log information of the operations of the work machine to a three-dimensional model of the work machine, and also sequentially generates movement track images that show, based on the operation information, movement tracks of a specific part of the work machine in association with the operation images; and a display control unit that causes to display the operation images overlapped with the movement track images on a designated display unit.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present disclosure relates to a display control system and a display control method.

Background Art

[0002] Patent Document 1 describes a playback device that enables the movement of two different construction machines (hereinafter referred to as work machines) to be grasped. The playback device described in Patent Document 1 reproduces the operation of a work machine by sequentially applying the angle information of the work machine included in the log information of the work machine to a 3D (three-dimensional) model of the work machine. In the playback device described in this Patent Document 1, the playback of the 3D model based on the first log information and the playback of the 3D model based on the second log information are performed synchronously.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in the analysis of work by a work machine, for example, there is a need to analyze a series of work flows such as excavation, turning from the excavation position to the loading position, loading, and turning from the loading position to the excavation position. According to the playback device described in Patent Document 1, such a series of movements can be grasped by, for example, repeatedly playing them back by animation using a 3D model. However, there is room for improvement in more effectively analyzing the work by a work machine.

[0005] The present disclosure has been made to solve the above problems, and an object thereof is to provide a display control system and a display control method that can more effectively analyze the work by a work machine.

Means for Solving the Problems

[0006] The display control system of the present disclosure includes an acquisition unit that acquires log information of a working machine associated with time, and an operation image representing the operation of the working machine is sequentially generated by sequentially applying operation information representing the operation of the working machine based on the log information to a three-dimensional model of the working machine. At the same time, a moving trajectory image representing the moving trajectory of a predetermined location of the working machine is sequentially generated based on the operation information in association with the operation image, and a display control unit that superimposes the operation image and the moving trajectory image and displays them on a predetermined display unit.

[0007] The display control method of the present disclosure includes a step of acquiring log information of a working machine associated with time, and an operation image representing the operation of the working machine is sequentially generated by sequentially applying operation information representing the operation of the working machine based on the log information to a three-dimensional model of the working machine. At the same time, a moving trajectory image representing the moving trajectory of a predetermined location of the working machine is sequentially generated based on the operation information in association with the operation image, and a step of superimposing the operation image and the moving trajectory image and displaying them on a predetermined display unit.

Advantages of the Invention

[0008] According to the display control system and the display control method of the present disclosure, the work by the working machine can be analyzed more effectively.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

[0010] <First Embodiment> Hereinafter, a display control system and a display control method according to the first embodiment will be described in detail with reference to FIGS. 1 to 14.

[0011] (Overall Configuration of Analysis Support System) FIG. 1 is a diagram showing the overall configuration of an analysis support system according to the first embodiment. The analysis support system 1 includes a reproduction system 10 and data loggers 20 mounted on each of a plurality of work machines 3. In this embodiment, the reproduction system 10 is an example of a configuration of the display control system according to the present disclosure.

[0012] The work machine 3 is the object of work analysis by the reproduction system 10. Examples of the work machine 3 include a hydraulic excavator, a wheel loader, a bulldozer, and the like. In the following description, a hydraulic excavator will be cited as an example of the work machine 3 for explanation. A plurality of sensors are provided on each work machine 3. The data logger 20 records and accumulates in time series the information indicating the state of the work machine 3 acquired by the sensors. Hereinafter, the information indicating the state of the work machine 3 at each time recorded by the data logger 20 will also be referred to as log information. In addition, when the operating mechanism for operating the work machine 3 is configured to operate the work machine 3 using an electrical operation signal, the information of the operation signal of the work machine 3 may be recorded and accumulated in time series and included in the log information. Further, the data logger 20 transmits the recorded log information to the reproduction system 10 via a wide area communication network at regular time intervals. Note that the regular time interval is, for example, a 5-minute interval. The reproduction system 10 records the log information received from the data logger 20 on a recording medium. The functions of the reproduction system 10 will be described later.

[0013] (Structure of the work machine) FIG. 2 is a diagram showing the structure of the work machine according to the first embodiment. The work machine 3, which is a hydraulic excavator, excavates and levels earth and sand at a work site or the like. As shown in FIG. 2, the work machine 3, which is a hydraulic excavator, includes a lower traveling body 31 for traveling and an upper revolving body 32 that is installed on the upper part of the lower traveling body 31 and can revolve. Further, the upper revolving body 32 is provided with a cab 32A, a work implement 32B, and two GNSS (Global Navigation Satellite System) antennas G1 and G2.

[0014] The lower traveling body 31 has a left crawler CL and a right crawler CR. The work machine 3 moves forward, turns, and moves backward by the rotation of the left crawler CL and the right crawler CR.

[0015] The operator's cab 32A is a place where the operator of the work machine 3 boards and performs operations. The operator's cab 32A is installed, for example, on the left side of the front end of the upper swing body 32. The configuration inside the operator's cab 32A will be described later.

[0016] The working machine 32B consists of a boom BM, an arm AR, and a bucket BK. The boom BM is attached to the front end of the upper swing body 32. An arm AR is attached to the boom BM. A bucket BK is attached to the arm AR. A boom cylinder SL1 is attached between the upper swing body 32 and the boom BM. By driving the boom cylinder SL1, the boom BM can be operated with respect to the upper swing body 32. An arm cylinder SL2 is attached between the boom BM and the arm AR. By driving the arm cylinder SL2, the arm AR can be operated with respect to the boom BM. A bucket cylinder SL3 is attached between the arm AR and the bucket BK. By driving the bucket cylinder SL3, the bucket BK can be operated with respect to the arm AR. The above-described upper swing body 32, boom BM, arm AR, and bucket BK provided in the work machine 3 which is a hydraulic excavator are one aspect of the movable parts of the work machine 3. Also, the bucket BK is a configuration example of the "working tool" according to the present disclosure. The bucket BK is provided with a cutting edge BKT used for excavation and the like.

[0017] (Configuration of the operator's cab) FIG. 3 is a diagram showing the configuration of the operator's cab of the work machine according to the first embodiment.

[0018] As shown in FIG. 3, the operator's cab 32A is provided with operation levers L1, L2, foot pedals F1, F2, and travel levers R1, R2. The operation lever L1 and the operation lever L2 are arranged on the left and right of the seat ST in the operator's cab 32A. Also, the foot pedals F1 and F2 are arranged on the floor surface in front of the seat ST in the operator's cab 32A.

[0019] An example of the operation pattern showing the correspondence between the input operations on the operation levers L1 and L2 and the travel levers R1 and R2 and the operations of the work machine 3, which is a hydraulic excavator, is as follows.

[0020] The operation lever L1 arranged on the left side toward the front of the cab is an operation mechanism for the slewing operation of the upper slewing body 32 and the excavation / dumping operation of the arm AR. Specifically, when the operator of the work machine 3 tilts the operation lever L1 forward, the arm AR performs a dumping operation. When the operator of the work machine 3 tilts the operation lever L1 backward, the arm AR performs an excavation operation. When the operator of the work machine 3 tilts the operation lever L1 to the right, the upper slewing body 32 slews to the right. When the operator of the work machine 3 tilts the operation lever L1 to the left, the upper slewing body 32 slews to the left. When the operation lever L1 is tilted in the front-rear direction, the upper slewing body 32 may slew to the right or left, and when the operation lever L1 is tilted in the left-right direction, the arm AR may perform a dumping operation or an excavation operation.

[0021] The operation lever L2 arranged on the right side toward the front of the cab is an operation mechanism for the excavation / dumping operation of the bucket BK and the raising / lowering operation of the boom BM. Specifically, when the operator of the work machine 3 tilts the operation lever L2 forward, the lowering operation of the boom BM is executed. When the operator of the work machine 3 tilts the operation lever L2 backward, the raising operation of the boom BM is executed. When the operator of the work machine 3 tilts the operation lever L2 to the right, the dumping operation of the bucket BK is performed. When the operator of the work machine 3 tilts the operation lever L2 to the left, the excavation operation of the bucket BK is performed.

[0022] Also, the travel levers R1 and R2 are operation mechanisms for controlling the operation of the lower travel body 31, that is, for controlling the travel of the work machine 3. The travel lever R1 arranged on the left side toward the front of the cab corresponds to the rotational drive of the left crawler CL of the lower travel body 31. Specifically, when the operator of the work machine 3 tilts the travel lever R1 forward, the left crawler CL rotates in the forward direction. When the operator of the work machine 3 tilts the travel lever R1 backward, the left crawler CL rotates in the reverse direction.

[0023] The travel lever R2 disposed on the right side toward the front of the driver's cab corresponds to the rotational drive of the right crawler CR of the lower travel body 31. Specifically, when the operator of the work machine 3 tilts the travel lever R2 forward, the right crawler CR rotates in the forward direction. Also, when the operator of the work machine 3 tilts the travel lever R2 backward, the right crawler CR rotates in the reverse direction. Note that the foot pedals F1 and F2 are interlocked with the travel levers R1 and R2 respectively, and travel control can also be performed by the foot pedals F1 and F2.

[0024] Note that the above-described operation pattern is only an example, and depending on the model of the hydraulic excavator etc., it is not limited to the above mode.

[0025] Note that depending on the embodiment, the work machine 3 described with reference to FIG. 2 may not be provided with the GNSS antennas G1 and G2.

[0026] (Functional configuration of the playback system) FIG. 4 is a diagram showing the functional configuration of the playback system according to the first embodiment. Hereinafter, with reference to FIG. 4, the functions of the playback system 10 according to the first embodiment will be described. As shown in FIG. 4, the playback system 10 includes a CPU 100, a memory 101, a display unit 102, an operation reception unit 103, a communication interface 104, and a storage 105. Note that the CPU (Central Processing Unit) 100 may be a processor such as an FPGA or a GPU instead of a CPU.

[0027] The CPU 100 is a processor that controls the overall operation of the playback system 10. Various functions of the CPU 100 will be described later.

[0028] The memory 101 is a so-called main memory device. In the memory 101, instructions and data necessary for the CPU 100 to operate based on a program are expanded.

[0029] The display unit 102 is a display device capable of visually displaying information, such as a liquid crystal display or an organic EL display.

[0030] The operation reception unit 103 is an input device, such as a general mouse, keyboard, touch sensor, etc.

[0031] The communication interface 104 is a communication interface for communicating with the data logger 20.

[0032] The storage 105 is a so-called auxiliary storage device, such as an HDD (Hard Disk Drive), SSD (Solid State Drive), etc. The storage 105 records the log information TL received from the data logger 20, the vehicle type of the working machine 3, the working machine model TM which is a 3D model prepared in advance for each model, etc. Note that the working machine model TM will be described later. In addition, the storage 105 also records the unit work prediction model PM1 which is a learned machine learning model used when estimating the work content of the working machine 3, the element work prediction model PM2, the heatmaps (H1, H2) generated during the estimation process, the estimated work content R of the working machine 3, the three-dimensional point cloud information MAP representing the terrain information of the work site, etc. Note that the unit work prediction model PM1, the element work prediction model PM2, and the heatmaps (H1, H2) will be described later. The three-dimensional point cloud information MAP includes, for example, three-dimensional point cloud information measured at the work site using a drone or the like, and information obtained by sequentially updating the terrain information from the cutting edge coordinate information of the working machine 3 during operation, and is recorded in the storage 105.

[0033] The functions of the CPU 100 of the reproduction system 10 according to the first embodiment will be described in detail. By operating based on a predetermined program, the CPU 100 exhibits functions as an acquisition unit 1000, a reception unit 1001, an extraction unit 1002, an image generation unit 1003, an estimation unit 1004, a determination unit 1005, a synchronization unit 1006, a display control unit 1007, and a designation unit 1008. Note that the above-described predetermined program may be for realizing a part of the functions to be exhibited by the playback system 10. For example, the program may exhibit functions by combining with other programs already stored in the storage 105 or by combining with other programs implemented in other devices. Note that in other embodiments, the playback system 10 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above-described configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, part or all of the functions realized by the processor may be realized by the integrated circuit.

[0034] The acquisition unit 1000 acquires log information TL to be played back from among a plurality of log information TLs recorded and stored in the storage 105. Here, it is assumed that the plurality of log information TLs are recorded file by file with different file names in the storage 105. The acquisition unit 1000 acquires, for example, two pieces of log information TL to be compared. Hereinafter, the case where the acquisition unit 1000 acquires two pieces of log information TL to be compared will be described as an example. One of the two pieces of log information acquired by the acquisition unit 1000 is log information designated by the operator as a playback target using a file name or the like. Hereinafter, this log information will be referred to as designated log information TL1 (first log information). The other of the two pieces of log information is log information that is played back simultaneously with the playback based on the designated log information TL1. Hereinafter, this log information will be referred to as comparison log information TL2 (second log information). The comparison log information TL2 may be, for example, log information pre-selected as a "model" of the work content for each of various work contents and conditions from viewpoints such as good fuel efficiency, short working hours, and driving by a skilled operator. Alternatively, the comparison log information TL2 may be, for example, past log information recorded in a previous operation performed by the same operator as the designated log information TL1. In this case, by comparing the two pieces of log information, for example, the proficiency of the operator can be analyzed.

[0035] The acquisition unit 1000 according to the present embodiment automatically searches for and acquires appropriate comparison log information TL2 as a comparison target for the designated log information TL1 based on the determination result by the determination unit 1005 described later. However, the acquisition unit 1000 according to other embodiments is not limited to the above aspect. For example, the acquisition unit 1000 may acquire the log information designated by the operator using a file name or the like as the comparison log information TL2. Further, when there is only one piece of recorded comparison log information TL2, the acquisition unit 1000 may acquire the one piece of comparison log information TL2.

[0036] The reception unit 1001 receives a predetermined reproduction instruction from an operator of the reproduction system 10. For example, the reception unit 1001 receives a reproduction instruction for the working machine 3 from an operator of the reproduction system 10.

[0037] The extraction unit 1002 extracts the angular information used for reproducing the working machine 3 from the acquired log information TL. The angular information is an example of the "operation information (motion information)" according to the present disclosure.

[0038] The image generation unit 1003 sequentially generates operation images representing the operation of the working machine 3 by sequentially applying operation information representing the operation of the working machine 3 based on the log information TL to the working machine model TM, which is a three-dimensional model of the working machine 3. At the same time, the image generation unit 1003 sequentially generates a movement trajectory image representing the movement trajectory of a predetermined location of the working machine 3 in association with the operation image based on the operation information. Further, the image generation unit 1003 can be configured to generate a movement trajectory image when the work content estimated by the estimation unit 1004 corresponds to the work content specified by the specification unit 1008. The movement trajectory is a path that starts from the position of the said location at the time obtained by going back a predetermined time from the time of the displayed operation image when the operation image is displayed as an animation, and passes through each position of the said location at each time when the time is advanced every predetermined time up to the current time, and ends at the position of the said location at the current time. The movement trajectory image can be, for example, a plurality of predetermined-shaped images representing those positions, such as spherical three-dimensional images, or linear or planar images connecting the positions. Also, the predetermined location of the working machine 3 for displaying the movement trajectory can be, for example, the cutting edge BKT of the bucket BK (working tool), or a predetermined position of the upper swing body 32. Also, for example, in other working machines 3, it can be the cutting edge of the blade, a predetermined position such as the grounding position of the crawler or tire, or the center coordinates of the working machine 3.

[0039] In addition, the image generation unit 1003 may generate a part image that associates and represents a predetermined part of the working machine 32B included in the working machine 3 at a predetermined time before with the operation image. The part image is a three-dimensional image representing the part at a time that goes back a predetermined time from the time of the operation image being displayed when the operation image is displayed as an animation. The predetermined part can be, for example, the bucket BK. In this case, the part image is a three-dimensional image representing the bucket BK, and can be displayed, for example, as a semi-transparent image, superimposed on the operation image.

[0040] In addition, the image generation unit 1003 changes aspects such as the display color of the movement trajectory image, the size and thickness of the image representing the movement trajectory, based on at least one of the work content estimated based on the log information TL, the accuracy of the position information included in the log information TL, the information indicating the presence or absence of automatic operation control in the working machine 3 included in the log information TL, the work amount of the working machine 3 based on the log information TL, and the information representing the work load of the working machine 3 based on the log information TL. Here, the accuracy of the position information may decrease or become unstable when the number of satellite acquisitions in GNSS is small, or when the baseline length to the reference point is long, and the display aspect of the movement trajectory can be changed between when the accuracy of the position information is good and when it is bad. As an example of an index of the accuracy of the position information, an RMS (Root Mean Square) value or a DOP (Dilution of Precision) value can also be used. Also, the automatic operation control is control that fully automatically controls or semi-automatically controls the operation of the working machine 3. The display aspect of the movement trajectory can be changed between when the automatic operation control is being performed and when it is not being performed. Also, the work amount of the working machine 3 is a value representing the magnitude, for example, of the excavation depth, the load during excavation, the weight of the load, etc. Also, the work load of the working machine 3 is a value indicating the height of the load applied to the work, calculated from, for example, the hydraulic pressure of the cylinder, the hydraulic pressure of the pump, the engine torque, the traction force, etc.

[0041] The estimation unit 1004 estimates the work content of the working machine 3 at each time from the acquired log information TL.

[0042] Based on the information included in the specified log information TL1, the determination unit 1005 determines whether the comparison log information TL2 is appropriate as a comparison target for the specified log information TL1.

[0043] The synchronization unit 1006 performs a process of synchronizing the reproduction of the machine tool model TM based on the specified log information TL1 and the reproduction of the machine tool model TM based on the comparison log information TL2. Specifically, the synchronization unit 1006 specifies the reproduction start time in the time series of the animation of the machine tool model TM based on the work content estimated by the estimation unit 1004.

[0044] The display control unit 1007 superimposes the operation image and the movement trajectory image generated by the image generation unit 1003 and displays them on the display unit 102. In addition, the display control unit 1007 superimposes the part image on the operation image and displays it on the display unit 102. Further, the display control unit 1007 may display an image (terrain image) representing the terrain based on the three-dimensional point cloud information MAP on the display unit 102.

[0045] The specifying unit 1008 specifies the work content to be the generation target of the movement trajectory image according to the operator's instruction. The specification of the work content can be performed, for example, by specifying by unit work, specifying by element work, specifying by a combination of unit work and element work, and the like.

[0046] (Processing Flow of the Reproduction System) Hereinafter, with reference to FIGS. 5 to 14, the specific processing flow performed by the reproduction system 10 will be described in detail.

[0047] The processing flow shown in FIG. 5 starts from the point in time when a dedicated application is launched by the operator of the reproduction system 10. When a dedicated application is launched by the operator's operation, the acquisition unit 1000 of the CPU 100 expands and acquires the specified log information TL1 specified as the reproduction target in the memory 101 (step S00).

[0048] Here, the log information TL (designated log information TL1, comparison log information TL2) will be described with reference to FIGS. 6 to 8.

[0049] As shown in FIGS. 6 to 8, the log information TL includes work machine identification information. Specifically, the work machine identification information is an individual identification number for individually identifying the work machine 3. In FIGS. 6 to 8, it is assumed that the work machine identification information is allocated so as to correspond to the vehicle type, model, type, machine number, etc. of the work machine 3 indicating a hydraulic excavator, wheel loader, bulldozer, etc. Note that the work machine identification information may be a number, letter, symbol, or a combination thereof other than a number.

[0050] As shown in FIG. 6, the log information TL includes information indicating the position and posture of the work machine 3 at each time, and angle information of the movable parts of the work machine 3. Specifically, in the log information TL, the position of the work machine 3, the roll angle of the work machine 3 which is the inclination in the left - right direction of the machine body, the pitch angle which is the inclination in the front - rear direction of the machine body, the turning angle, the boom angle, the arm angle, and the bucket angle are recorded for each time. Here, the data logger 20 mounted on the work machine 3 is, for example, information obtained by receiving GNSS antennas G1, G2, and based on the positioning information indicating latitude, longitude, and altitude, identifies and records the position of the work machine 3. Further, the data logger 20 calculates and records the roll angle and pitch angle of the work machine 3 based on the measurement results of an IMU (Inertial Measurement Unit) mounted on the work machine 3. Also, the data logger 20 calculates and records the turning angle of the upper swing body 32 based on the positioning information obtained from each of the GNSS antennas G1, G2 provided on the upper swing body 32. Furthermore, the data logger 20 calculates and records the boom angle, arm angle, and bucket angle based on the extension / contraction degrees of the boom cylinder SL1, arm cylinder SL2, and bucket cylinder SL3, respectively. Note that the boom angle, arm angle, bucket angle, and turning angle may be obtained, for example, by attaching an IMU to the boom, arm, bucket, and upper swing body and using those IMUs.

[0051] Note that the position, roll angle, and pitch angle are information necessary to specify the position and orientation of the working machine 3 itself. Therefore, for example, in an embodiment where only the movements of the movable parts of the working machine 3, that is, the upper swing body 32, the boom BM, the arm AR, and the bucket BK are animated and the position and orientation of the working machine 3 itself are not reproduced, the information on the position, roll angle, and pitch angle does not need to be included in the log information.

[0052] The log information TL shown in FIG. 6 corresponds to the "operation information" according to the present disclosure. Alternatively, the log information TL shown in FIG. 6 is information representing the operation of the working machine 3 that can be visually recognized from the outside of the working machine 3, and can also be referred to as appearance operation information. That is, in the present embodiment, the operation information or the appearance operation information includes the position information, roll angle, pitch angle, swing angle, boom angle, arm angle, and bucket angle of the working machine 3. In the present embodiment, different from the drive mechanism information and operation information described later, the operation information or the appearance operation information has the feature of being less affected by the difference in the vehicle type of the working machine 3 as information used when estimating the work content.

[0053] Also, as shown in FIG. 7, the log information TL includes the degree of input to the operator's operation levers L1, L2, etc. at each time, that is, the PPC pressure, which is the pilot hydraulic pressure indicating the degree of tilt of the lever and the degree of depression of the pedal. Specifically, the log information TL records the PPC pressures of the operation levers L1, L2, the travel levers R1, R2, or the foot pedals F1, F2 corresponding to each operation type of left / right swing, arm excavation / dumping, boom raising / lowering, bucket excavation / dumping, right crawler forward / backward, and left crawler forward / backward by the operator at each time. Note that each time shown in FIG. 7 corresponds to each time in FIG. 6. In the present embodiment, each piece of information shown in FIG. 7 is also referred to as operation information.

[0054] Further, as shown in FIG. 8, the log information TL includes information indicating the status of the main drive mechanisms such as the engine and hydraulic pump of the working machine 3 at each time. Specifically, in the log information TL, the engine coolant temperature, engine output, instantaneous fuel consumption, and oil temperature of the hydraulic pump are recorded for each time. Note that each time shown in FIG. 8 corresponds to each time in FIGS. 6 and 7. In the present embodiment, each piece of information shown in FIG. 8 is also referred to as drive mechanism information.

[0055] Returning to FIG. 5, the estimation unit 1004 of the CPU 100 estimates the work content of the working machine 3 at each time based on the designated log information TL1 acquired in step S00 (step S01).

[0056] Here, the procedure for the estimation unit 1004 to estimate the work content of the working machine 3 from the log information TL will be described with reference to FIG. 9. The estimation unit 1004 estimates the work content of the working machine 3 with respect to both unit work and elemental work. Unit work is work that accomplishes one work objective. Elemental work is work that represents a series of operations or work that constitutes unit work and is classified by purpose.

[0057] Examples of the classification of unit work include, for example, "excavation and loading", "weeding", "slope (from below)", "loading collection", "traveling", "stopping and resting", as shown in FIG. 9, and also "ditch excavation", "backfilling", "slope (from above)", etc. Excavation and loading is work that digs, cuts, and loads the excavated earth and sand or rock onto the loading platform of a transport vehicle. Excavation and loading is a unit work composed of excavation, loading swing, soil discharge, empty swing, waiting for soil discharge, and loading platform pressing. Weeding is work that flattens and cuts off the excess undulations on the ground to a predetermined height. Weeding is a unit work composed of excavation and soil discharge, or excavation, loading swing, soil discharge, and empty swing, and may include leveling and sweeping. Slope (from below) is work that creates a slope by the working machine 3 located below the target location. Slope (from below) is a unit work composed of rolling compaction, excavation, loading swing, soil discharge, and empty swing, and may include leveling. Loading is the operation of collecting the earth and sand excavated by excavation or the like before loading it onto a transport vehicle. Loading consists of excavation, loading swing, soil discharge, and empty swing, and is a unit operation that may include leveling. Traveling is the operation of moving the work machine 3. Traveling as a unit operation is a unit operation composed of traveling as an elemental operation. Stopping and idling is a state where there is no earth and sand or rock in the bucket BK and the machine has stopped for a predetermined time or more. Stopping and idling as a unit operation is a unit operation composed of stopping as an elemental operation. Trench excavation is the operation of digging and scraping the ground into a long and narrow groove shape. Trench excavation consists of excavation, loading swing, soil discharge, and empty swing, and is a unit operation that may include leveling. Backfilling is the operation of putting earth and sand into an existing groove or hole in the ground and filling it flat. Backfilling consists of excavation, loading swing, soil discharge, compaction, and empty swing, and is a unit operation that may include leveling and sweeping. Slope formation (from above) is the operation of creating a slope by the work machine 3 located above the target location. Slope formation (from above) consists of compaction, excavation, loading swing, soil discharge, and empty swing, and is a unit operation that may include leveling.

[0058] Examples of the classification of elemental operations include "excavation", "loading swing", "waiting for soil discharge", "soil discharge", "empty swing", "holding down the loading platform" shown in FIG. 9, and also "compaction", "leveling", "sweeping", etc. Excavation is the operation of digging and scraping earth and sand or rock with the bucket BK. Loading swing is the operation of swinging the upper slewing body 32 while holding the excavated earth and sand or rock in the bucket BK. Waiting for soil discharge is the operation of waiting for a transport vehicle for loading while holding the excavated earth and sand or rock in the bucket BK. Soil discharge is the operation of dropping the excavated earth and sand or rock from the bucket BK into a transport vehicle or a predetermined location. Empty swing is the operation of swinging the upper slewing body 32 with no earth and sand or rock in the bucket BK. The bed pressing is an operation of pressing the earth and sand loaded on the bed of the transport vehicle from above with the bucket BK to make it flat. The rolling compaction is an operation of pushing the earth and sand into the disturbed ground with the bucket BK to form and strengthen the ground. The leveling is an operation of leveling the earth and sand with the bottom surface of the bucket BK. The sweeping is an operation of leveling the earth and sand with the side surface of the bucket BK.

[0059] The estimation unit 1004 obtains a time series of likelihoods related to the unit operations by inputting the log information TL into the unit operation prediction model PM1 in chronological order. The unit operation prediction model PM1 is a model that outputs the likelihood related to the unit operations when the log information TL is input, for example, by learning using teacher data, and may be stored in the storage 105, for example.

[0060] Also, the estimation unit 1004 obtains a time series of likelihoods related to the elemental operations by inputting the log information TL into the elemental operation prediction model PM2 in chronological order. The elemental operation prediction model PM2 is a model that outputs the likelihood related to the elemental operations when the log information TL is input, for example, by learning using teacher data, and may be stored in the storage 105, for example.

[0061] The estimation unit 1004 smooths the time series of likelihoods by applying the time series of likelihoods related to the unit operations and the time series of likelihoods related to the elemental operations to the time average filter respectively, and generates a unit operation heat map H1 representing the smoothed time series of likelihoods related to the unit operations and an elemental operation heat map H2 representing the smoothed time series of likelihoods related to the elemental operations as shown in FIG. 9. The heat maps H1 and H2 are maps in which, based on the smoothed time series of likelihoods, a color representing the likelihood of the operation category is assigned to a plane with the operation category on the vertical axis and the time on the horizontal axis. The color related to the heat map may approach blue as the likelihood of the operation category is lower, and approach red as the likelihood of the operation category is higher, for example. The estimation unit 1004 stores the heat maps H1 and H2 in the storage 105.

[0062] Based on the smoothed likelihood time series, the estimation unit 1004 identifies the time period when the likelihood of the unit operation is dominant and estimates the operation content of the working machine 3 during that time period. For example, if the time period when the likelihood of the unit operation "excavation and loading" is dominant, the operation content of the working machine 3 is estimated as "excavation and loading". Similarly, based on the smoothed likelihood time series, the estimation unit 1004 identifies the time period when the likelihood of the elemental operation is dominant and estimates the operation content of the working machine 3 during that time period. For example, if the time period when the likelihood of the elemental operation "excavation" is dominant, the operation content of the working machine 3 is estimated as "excavation". The estimation unit 1004 stores the information on the operation content R estimated for the working machine 3 in the storage 105.

[0063] Returning to FIG. 5, subsequently, the acquisition unit 1000 acquires comparison log information TL2 appropriate as a comparison target for the designated log information TL1 acquired in step S00 (step S02). At this time, the determination unit 1005 performs a determination processing flow for determining whether the plurality of comparison log information TL2 recorded in the storage 105 is appropriate as a comparison target for the designated log information TL1 acquired in step S00. In this determination processing flow, first, the determination unit 1005 selects one from among the plurality of comparison log information TL2 previously recorded in the storage 105. Then, based on the log information TL, the determination unit 1005 determines, for example, whether the vehicle compartment, the change amount of the turning angle (the magnitude of the swing), the bucket height when the working machine 3 is waiting for soil discharge, etc. are of the same degree. If it is determined that they are not of the same degree, a new comparison log information TL2 is selected, and the same determination processing as above is performed. In addition, the determination unit 1005 according to another embodiment may narrow down the candidates for reproduction based on the working position, working time, classification of the work, etc. in addition to the above determination processing.

[0064] Returning to FIG. 5, next, the synchronization unit 1006 performs a process of synchronizing the animation playback of the machine model TM based on the specified log information TL1 and the animation playback of the machine model TM based on the comparison log information TL2. Specifically, the synchronization unit 1006 identifies the playback start time in the time series in the animation of the machine model TM by the specified log information TL1 and the playback start time in the time series in the animation of the machine model TM by the comparison log information TL2.

[0065] The method for identifying the playback start time for synchronizing the two is as follows. That is, the synchronization unit 1006 extracts the timing of the switching of the unit operation in the specified log information TL1 based on the unit operation heat map H1 for the specified log information TL1 generated by the estimation unit 1004. Next, the synchronization unit 1006 extracts the timing of the switching of the unit operation in the comparison log information TL2 based on the unit operation heat map H1 for the comparison log information TL2 generated by the estimation unit 1004. The synchronization unit 1006 selects one of the extracted timings and identifies it as the playback start time of each animation. Note that the playback system 10 according to other embodiments is not limited to the above aspect. The playback system 10 (synchronization unit 1006) according to other embodiments may, for example, compare the combination of the angle information of the boom, arm, bucket, etc. in the comparison log information TL2 with the combination of the angle information of the boom, arm, bucket, etc. in the specified log information TL1, extract the timing when both are at close angles, and identify this as the playback start time of each animation.

[0066] Next, the reception unit 1001 receives a playback instruction from the operator (step S04). As one aspect of the reproduction instruction, it may be an operation such as pressing a play button. Further, a reproduction instruction may be given including information such as time, the position of the working machine 3, and various events such as an abnormality occurring in the working machine 3, which serve as the starting point of reproduction. Further, when the reception unit 1001 receives a reproduction instruction, the designation unit 1008 may designate the work content to be the generation target of the movement locus image.

[0067] Next, the acquisition unit 1000 selects and reads out a working machine model TM corresponding to the reference working machine identification information from the storage 105 based on the working machine identification information as the type of the working machine 3 received by the reception unit 1001 (step S05).

[0068] Here, the working machine model TM will be described with reference to FIG. 10. As shown in FIG. 10, the working machine model TM is information including working machine identification information and an outer shape 3D model M0 of the working machine 3 indicated by the working machine identification information. The outer shape 3D model M0 is a 3D model representing the working machine 3, and is constructed for each part of the working machine 3 such as a lower traveling body and an upper slewing body. For example, the outer shape 3D model M0 represents the shape of the working machine 3. For example, the outer shape 3D model M0 includes a lower traveling body outer shape model M01 representing the lower traveling body 31 of the working machine 3, an upper slewing body outer shape model M02 representing the upper slewing body 32, a boom outer shape model M03 representing the boom BM, an arm outer shape model M04 representing the arm AR, and a bucket outer shape model M05 representing the bucket BK.

[0069] Returning to FIG. 5, the extraction unit 1002 extracts information for reproduction from each of the designated log information TL1 and the comparison log information TL2 (step S06). For example, the extraction unit 1002 extracts various angle information such as boom angle, arm angle, and bucket angle as information for reproduction. Note that the pilot hydraulic pressure shown in FIG. 7 may be extracted as information for reproduction. Further, in step S00 and step S02, only the information used for reproduction may be acquired.

[0070] Next, the image generation unit 1003 and the display control unit 1007 execute simultaneous playback of the animation of the construction machine model TM based on the designated log information TL1 and the playback of the animation of the construction machine model TM based on the comparison log information TL2 (step S07). Here, the image generation unit 1003 applies various information recorded in the designated log information TL1 to the construction machine model TM in the order of oldest time stamps from the playback start time specified in the synchronization process of step S03, while generating an operation image representing the operation of the construction machine 3 (step S07a). Further, simultaneously with the generation of the operation image for this animation, the image generation unit 1003 applies various information recorded in the comparison log information TL2 to the construction machine model TM in the order of oldest time stamps from the playback start time specified in the synchronization process of step S03, while generating an operation image representing the operation of the construction machine 3 (step S07a).

[0071] Hereinafter, the content of the operation image generation process by the image generation unit 1003 will be described in detail. The image generation unit 1003 changes the angle of the corresponding part of the outer shape 3D model M0 based on various angle information such as the turning angle and boom angle shown in the log information TL (designated log information TL1, comparison log information TL2). For example, the image generation unit 1003 tilts the bucket outer shape model M05 around the rotation axis defined at the connection position with the arm outer shape model M04 to the bucket angle shown in the log information TL, thereby reproducing the position and posture of the bucket BK of the construction machine 3.

[0072] Similarly, the image generation unit 1003 tilts the arm outer shape model M04 around the rotation axis defined at the connection position with the boom outer shape model M03 to the arm angle shown in the log information TL, thereby reproducing the position and posture of the arm AR of the construction machine 3.

[0073] Similarly, the image generation unit 1003 tilts the upper swing body outer shape model M02 around the rotation axis defined at the connection position with the lower traveling body outer shape model M01 to the swing angle shown in the log information TL, thereby reproducing the position and posture of the upper swing body 32 of the construction machine 3.

[0074] Similarly, the image generation unit 1003 tilts the lower traveling body external shape model M01 around the roll rotation axis defined in the lower traveling body external shape model M01 to the roll angle shown in the log information TL, and tilts it around the pitch rotation axis defined in the lower traveling body external shape model M01 to the pitch angle shown in the log information TL, thereby reproducing the posture of the lower traveling body 31 of the working machine 3.

[0075] In addition, the reproduction system 10 according to the first embodiment can reproduce the traveling animation of the working machine 3 based on the PPC pressures of the right crawler forward / backward and the left crawler forward / backward included in the log information TL at each time.

[0076] Specifically, based on the PPC pressures of the right crawler forward / backward and the left crawler forward / backward, the external shape 3D model M0 moves forward, backward, left and right forward, and left and right backward. For example, based on the numerical values of the PPC pressures of the right crawler forward and the left crawler forward, the external shape 3D model M0 is moved in the forward direction. The moving speed may be changed based on the numerical values of the PPC pressures.

[0077] In addition, based on the numerical values of the PPC pressures of the right crawler backward and the left crawler backward, the external shape 3D model M0 is moved in the backward direction. Also, based on the difference in the numerical values of the PPC pressures of the right crawler forward and the left crawler forward, the external shape 3D model M0 is moved so as to curve in the forward left and right directions. For example, when the numerical value of the PPC pressure of the right crawler forward is larger than the numerical value of the PPC pressure of the left crawler forward, it is moved so as to curve in the forward left direction. The moving speed and the curvature size may be changed according to the respective numerical values of the PPC pressures and the difference in the numerical values of the PPC pressures.

[0078] Similarly, based on the difference in the numerical values of the PPC pressures of the right crawler backward and the left crawler backward, the external shape 3D model M0 is moved so as to curve in the backward left and right directions. For example, when the numerical value of the PPC pressure of the right crawler backward is larger than the numerical value of the PPC pressure of the left crawler backward, it is moved so as to curve in the backward left direction. The moving speed and the curvature size may be changed according to the respective numerical values of the PPC pressures and the difference in the numerical values of the PPC pressures.

[0079] In addition to the PPC pressures for the right crawler forward / backward and the left crawler forward / backward, by using the position information for playback, the running of the work machine 3 can be animated more accurately. In this case, by using the position information, the speed and position of the movement of the work machine 3 can be expressed more accurately. Further, in addition to the PPC pressures for the right crawler forward / backward and the left crawler forward / backward, based on the roll angle, or the pitch angle, or both the roll angle and the pitch angle, by animating the work machine 3, the lateral inclination of the work machine 3 during running or the longitudinal inclination of the work machine 3 can be reproduced.

[0080] Also, in step S07, the image generation unit 1003 sequentially generates a movement trajectory image representing the movement trajectory of a predetermined position (the cutting edge BK of the bucket BK) of the work machine 3 in association with the operation image based on the operation information (step S07b). Further, the display control unit 1007 superimposes the operation image and the movement trajectory image and causes the display unit 102 to display them. Also, in step S07, the image generation unit 1003 sequentially generates a part image representing a predetermined part (the bucket BK) of the work machine 32B in association with the operation image (step S07c). Also, in step S07c, the display control unit 1007 superimposes the operation image, the movement trajectory image, and the part image and causes the display unit 102 to display them. Also, in step S07, the display control unit 1007 synthesizes a terrain image based on the three-dimensional point cloud information MAP with the operation image, the movement trajectory image, and the part image and causes the display unit 102 to display them (step S07d). FIG. 11 schematically shows a display image D1 displayed by the display control unit 1007 on the display unit 102. The display image D1 includes an operation image D11 based on the designated log information TL1, a movement trajectory image D12, a part image D13, and a terrain image D14. FIG. 12 schematically shows a display image D2 displayed by the display control unit 1007 on the display unit 102. The display image D2 includes, in addition to the operation image D11 based on the designated log information TL1, the movement trajectory image D12, the part image D13, and the terrain image D14, an operation image D21 based on the comparison log information TL2, a movement trajectory image D22, and a part image D23.

[0081] During the animation playback of the operations of the two working machines 3, the image generation unit 1003 determines whether to end the animation playback (step S08). For example, when receiving an instruction to end the playback based on pressing a stop button or the like, it is determined to end the animation playback. After the start of the animation playback, it may be determined to end the animation playback after a predetermined period has elapsed. If the animation playback does not end (step S08; NO), the image generation unit 1003 continues the simultaneous animation playback of the two working machine models TM. On the other hand, when ending the animation playback (step S08; YES), the image generation unit 1003 ends the animation playback process.

[0082] Among the processing flows described with reference to FIG. 5, steps S00, S01, S03, S04, S05, S06, and S08 are not essential components of the playback system 10, and in other embodiments, such steps may not be provided.

[0083] (Other display examples) FIG. 13 schematically shows a display image D3 displayed by the display control unit 1007 on the display unit 102. The display image D3 includes an operation image D31 based on the log information TL1, a movement trajectory image D32, and a terrain image D34. The movement trajectory image D32 shown in FIG. 13 has a changing display mode, for example, a changing display color, based on the work content. FIG. 14 schematically shows a display image D4 displayed by the display control unit 1007 on the display unit 102. The display image D4 includes an operation image D41 based on the log information TL1, a movement trajectory image D42, and a terrain image D44. The movement trajectory image D42 shown in FIG. 14 has a changing display mode, for example, a changing display color, based on the accuracy of the position information.

[0084] (Function, effect) As described above, the playback system 10 according to the first embodiment includes an acquisition unit 1000 that acquires the log information TL of the working machine 3 associated with time, and sequentially applies operation information representing the operation of the working machine 3 based on the log information TL to the three-dimensional model of the working machine 3 to sequentially generate operation images D11 and D21 representing the operation of the working machine 3. At the same time, a moving trajectory image D12 and D22 representing the moving trajectory of a predetermined position of the working machine 3 associated with the operation image are sequentially generated based on the operation information by an image generation unit 1003, and a display control unit 1007 that superimposes the operation image and the moving trajectory image and displays them on a predetermined display unit 102. According to this configuration, since the flow of a series of operations can be easily grasped by the moving trajectory image, the operation by the working machine 3 can be analyzed more effectively.

[0085] (Modification example) The content of the log information TL according to the first embodiment (Figs. 6 to 8) is not limited to this in other embodiments. For example, when the working machine 3 is not a hydraulic excavator but another vehicle type, log information TL corresponding to the vehicle type is recorded. Other vehicle types are, for example, wheel loaders, bulldozers, etc.

[0086] Also, the playback system 10 according to the first embodiment has been described as being installed at a location away from the working machine 3 and connected to the data logger 20 mounted on the working machine 3 via a wide area communication network, but is not limited to this aspect in other embodiments.

[0087] For example, in the playback system 10 according to other embodiments, part or all of the configuration of the playback system 10 may be installed inside the working machine 3. In this case, the data logger 20 may transmit the log information TL to the playback system 10 via a network inside the working machine 3 without using a wide area communication network. By doing so, the operator boarding the working machine 3 can play back and confirm the movement of the working machine 3 operated by the operator himself / herself on the spot. Also, by playing back the movement of the working machine 3 as a model for the operator of the working machine 3, it can be used as guidance.

[0088] Note that the playback system 10 installed inside the working machine 3 may acquire the log information TL of other working machines 3 via a wide - area communication network or the like. By doing so, the state of the working machine 3 other than the working machine 3 equipped with the playback system 10 can be animated and played back.

[0089] Also, the playback system 10 according to another embodiment may be configured to transmit the video information generated by the animation playback process to the monitor mounted on the working machine 3 for display while being installed at a location away from the working machine 3.

[0090] In another embodiment, as one aspect of the playback instruction received from the operator, for example, it may be to specify the playback period. For example, the playback period may be the playback start time and the playback end time. In this case, the playback system 10 performs the playback of the working machine 3 for the received playback period. Also, in another embodiment, the specification of the playback end time is not essential. For example, in another embodiment, as a playback instruction from the operator, it may be a mode of receiving only the playback start time and playing back for a certain period from the playback start time, or a mode of continuing to play back as long as the log information exists, or a mode of stopping the playback in accordance with the occurrence of various other events.

[0091] Note that the log information TL (Figs. 6 - 8) to be acquired does not have to be arranged in chronological order. In this case, the image generation unit 1003 may apply the information used for playback among the log information TL to the working machine model TM in chronological order.

[0092] Incidentally, the processes of various processes of the above-described playback system 10 are stored in a computer-readable recording medium in the form of a program, and the above-described various processes are performed by the computer reading and executing this program. Further, the computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Further, this computer program may be distributed to a computer via a communication line, and the computer that has received this distribution may execute the program.

[0093] The above program may be for realizing a part of the above-described functions. Further, it may be a so-called difference file, a difference program, or the like that can realize the above-described functions in combination with a program already recorded in a computer system.

[0094] A part or all of the functions of the above-described playback system 10 may be assigned to the machine tool 3. For example, some or all of the functions of the acquisition unit 1000, the reception unit 1001, the extraction unit 1002, the image generation unit 1003, the estimation unit 1004, the determination unit 1005, the synchronization unit 1006, the display control unit 1007, the designation unit 1008, the memory 101, the display unit 102, the operation reception unit 103, the communication interface 104, and the storage 105 may be assigned to the machine tool 3.

[0095] As described above, some embodiments of the present disclosure have been described, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalent scope thereof.

[0096] (Supplementary Note) The display control system (playback system 10) according to the present disclosure is grasped as follows, for example.

[0097] (1) The display control system (reproduction system 10) according to the first aspect includes an acquisition unit 1000 that acquires log information TL of the working machine 3 associated with time, and operation information representing the operation of the working machine based on the log information is sequentially applied to a three-dimensional model (working machine model TM) of the working machine to sequentially generate operation images (D11, D21, D31, D41) representing the operation of the working machine, and a movement trajectory image (D12, D22, D32, D42) representing the movement trajectory of a predetermined portion of the working machine is sequentially generated based on the operation information in association with the operation image. An image generation unit 1003 and a display control unit 1007 that superimposes the operation image and the movement trajectory image and displays them on a predetermined display unit 102. According to this aspect and the following aspects, the work by the working machine can be analyzed more effectively.

[0098] (2) The display control system (reproduction system 10) according to the second aspect is the display control system of (1), and further includes an estimation unit 1004 that estimates the work content of the working machine at each time based on the log information, and a designation unit 1008 that designates the work content to be the generation target of the movement trajectory image. The image generation unit 1003 generates the movement trajectory image when the estimated work content corresponds to the designated work content.

[0099] (3) The display control system (reproduction system 10) according to the third aspect is the display control system of (1) or (2), and the image generation unit 1003 generates a part image (D13, D23) representing a predetermined part of a working tool included in the working machine at a predetermined time in association with the operation image, and the display control unit 1007 superimposes the part image on the operation image and displays it on the display unit.

[0100] (4) The display control system (reproduction system 10) according to the fourth aspect is the display control system of (1) to (3), and the predetermined portion corresponds to the cutting edge of a working tool included in the working machine.

[0101] (5) The display control system (reproduction system 10) according to the fifth aspect is the display control system of (1) to (3), and the predetermined position corresponds to the grounding position of the crawler or tire provided in the work machine.

[0102] (6) The display control system (reproduction system 10) according to the sixth aspect is the display control system of (1) to (5), and the image generation unit changes the mode of the movement trajectory image based on at least one of the work content estimated based on the log information, the accuracy of the position information included in the log information, the information indicating the presence or absence of automatic operation control in the work machine included in the log information, the work amount of the work machine based on the log information, and the information indicating the work load of the work machine based on the log information.

Explanation of Signs

[0103] 1 Analysis support system, 10 Reproduction system, 100 CPU, 1000 Acquisition unit, 1001 Reception unit, 1002 Extraction unit, 1003 Image generation unit, 1004 Estimation unit, 1005 Judgment unit, 1006 Synchronization unit, 1007 Display control unit, 1008 Designation unit, 101 Memory, 102 Display unit, 103 Operation reception unit, 104 Communication interface, 105 Storage, 20 Data logger, 3 Work machine, H1~H2 Heat map, TM Work machine model, TL Log information, PM1 Unit work prediction model, PM2 Element work prediction model, R Estimated work content, D11, D21, D31, D41 Action image, D12, D22, D32, D42 Movement trajectory image, D13, D23 Part image

Claims

1. An acquisition unit that acquires log information of a working machine associated with a time; An image generation unit that sequentially generates operation images representing the operations of the working machine by sequentially applying operation information representing the operations of the working machine based on the log information to a three-dimensional model of the working machine, and sequentially generates movement trajectory images representing the movement trajectories of predetermined locations of the working machine in association with the operation images based on the operation information; A display control unit that superimposes the operation image and the movement trajectory image and displays them on a predetermined display unit A display control system comprising.

2. An estimation unit that estimates the work content of the working machine at each time based on the log information; A designation unit that designates the work content to be the generation target of the movement trajectory image; Further comprising, The image generation unit generates the movement trajectory image when the estimated work content corresponds to the designated work content The display control system according to claim 1.

3. The image generation unit generates a part image representing a predetermined part of a working tool included in the working machine before a predetermined time in association with the operation image; The display control unit superimposes the part image on the operation image and displays it on the display unit The display control system according to claim 2.

4. The predetermined location corresponds to the tip of a working tool included in the working machine The display control system according to claim 3.

5. The predetermined location corresponds to the ground contact position of a crawler or tire included in the working machine The display control system according to claim 3.

6. The image generation unit changes the mode of the movement trajectory image based on at least one of the work content estimated based on the log information, the accuracy of the position information included in the log information, the information indicating the presence or absence of automatic operation control in the working machine included in the log information, the work amount of the working machine based on the log information, and the information representing the work load of the working machine based on the log information The display control system according to claim 4.

7. A step of acquiring log information of a working machine associated with a time; A step of sequentially generating operation images representing the operations of the working machine by sequentially applying operation information representing the operations of the working machine based on the log information to a three-dimensional model of the working machine, and sequentially generating movement trajectory images representing the movement trajectories of predetermined locations of the working machine in association with the operation images based on the operation information; A step of superimposing the motion image and the movement trajectory image and displaying them on a predetermined display unit A display control method including the above

Images