Replay system and replay method

Abstract

To replay the movements of a work machine in an easily graspable manner.SOLUTION: A replay system comprises: an acquisition unit that acquires chronically-mapped log information of a work machine; an image generation unit that generates operation images of operations of the work machine by sequentially applying operation information based on the log information of the operation of the work machine to a three-dimensional model of the work machine; and a display control unit that causes to display the operation images on a designated display unit. The three-dimensional model includes an operation unit model that shows an operation unit arranged in the driving cab of the work machine. The image generation unit generates the operation images with the three-dimensional model being set at a view point facing the operation unit in the driving cab.SELECTED DRAWING: Figure 14

Description

Technical Field

[0001] The present disclosure relates to a playback system and a playback method.

Background Art

[0002] Patent Document 1 describes a playback device that enables the movement of two different construction machines (hereinafter referred to as work machines) to be grasped. The playback device described in Patent Document 1 reproduces the operation of the work machine by sequentially applying the angle information of the work machine included in the log information of the work machine to the 3D (three-dimensional) model of the work machine. In the playback device described in Patent Document 1, the playback of the 3D model based on the first log information and the playback of the 3D model based on the second log information are performed synchronously. Further, in the playback device described in Patent Document 1, using a two-dimensional operation panel model that represents changes in the operation amount by movements in the vertical and horizontal directions perpendicular to each other, etc., the input operations on various operation levers and travel levers by the operator of the work machine are animated and played back.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described above, in the playback device described in Patent Document 1, the input operations on the operation lever and the travel lever are animated and played back using a two-dimensional operation panel model. However, there is room for improvement in making it easier to grasp and reproduce the movement of the work machine.

[0005] The present disclosure has been made to solve the above problems, and an object thereof is to provide a playback system and a playback method that can make it easier to grasp and reproduce the movement of a work machine.

Means for Solving the Problem

[0006] The playback system of the present disclosure includes an acquisition unit that acquires log information of a working machine associated with a time, an image generation unit that generates an operation image representing the operation of the working machine by sequentially applying operation information representing the operation of the working machine based on the log information to a three-dimensional model of the working machine, and a display control unit that causes the operation image to be displayed on a predetermined display unit. The three-dimensional model includes an operation unit model representing an operation unit provided in the driver's cab of the working machine. The image generation unit generates the operation image by setting the three-dimensional model at a viewpoint facing the operation unit in the driver's cab.

[0007] The playback method of the present disclosure includes a step of acquiring log information of a working machine associated with a time, a step of generating an operation image representing the operation of the working machine by sequentially applying operation information representing the operation of the working machine based on the log information to a three-dimensional model of the working machine, and a step of causing the operation image to be displayed on a predetermined display unit. The three-dimensional model includes an operation unit model representing an operation unit provided in the driver's cab of the working machine. When generating the operation image, the three-dimensional model is set at a viewpoint facing the operation unit in the driver's cab to generate the operation image.

Advantages of the Invention

[0008] According to the playback system and the playback method of the present disclosure, it is possible to easily grasp and reproduce the movement of the working machine.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Modes for Carrying Out the Invention

[0010] <The First Embodiment> Hereinafter, a display control device and a display control method according to the first embodiment will be described in detail with reference to FIGS. 1 to 18.

[0011] (Overall Configuration of Analysis Support System) FIG. 1 is a diagram showing the overall configuration of the analysis support system according to the first embodiment. The analysis support system 1 includes a playback system 10 and data loggers 20 mounted on each of a plurality of working machines 3.

[0012] The working machine 3 is the object of work analysis by the playback system 10. Examples of the working machine 3 include a hydraulic excavator, a wheel loader, a bulldozer, etc. In the following description, a hydraulic excavator will be cited as an example of the working machine 3 for explanation. A plurality of sensors are provided on each working machine 3. The data logger 20 records and accumulates in time series the information indicating the state of the working machine 3 acquired by the sensors. Hereinafter, the information indicating the state of the working machine 3 at each time recorded by the data logger 20 will also be referred to as log information. In addition, when the operating mechanism for operating the working machine 3 is configured to operate the working machine 3 using an electrical operation signal, the information of the operation signal of the working machine 3 may be recorded and accumulated in time series and included in the log information. Further, the data logger 20 transmits the recorded log information to the playback system 10 via a wide area communication network at regular time intervals. Note that the regular time interval is, for example, a 5-minute interval. The playback system 10 records the log information received from the data logger 20 on a recording medium. The functions of the playback system 10 will be described later.

[0013] (Structure of Working Machine) FIG. 2 is a diagram showing the structure of the working machine according to the first embodiment. The working machine 3, which is a hydraulic excavator, excavates and levels earth and sand at a work site or the like. As shown in FIG. 2, the working machine 3, which is a hydraulic excavator, includes a lower traveling body 31 for traveling and an upper revolving body 32 installed on the upper part of the lower traveling body 31 and capable of revolving. In addition, the upper revolving body 32 is provided with a cab 32A, a working device 32B, and two GNSS (Global Navigation Satellite System) antennas G1, G2.

[0014] The lower traveling body 31 has a left crawler CL and a right crawler CR. The working machine 3 moves forward, turns, and moves backward by the rotation of the left crawler CL and the right crawler CR.

[0015] The driver's cab 32A is a place where the operator of the working machine 3 boards and performs operations. The driver's cab 32A is installed, for example, on the left side of the front end of the upper slewing body 32. The internal configuration of the driver's cab 32A will be described later.

[0016] The working implement 32B consists of a boom BM, an arm AR, and a bucket BK. The boom BM is attached to the front end of the upper slewing body 32. An arm AR is attached to the boom BM. A bucket BK is attached to the arm AR. A boom cylinder SL1 is attached between the upper slewing body 32 and the boom BM. By driving the boom cylinder SL1, the boom BM can be operated with respect to the upper slewing body 32. An arm cylinder SL2 is attached between the boom BM and the arm AR. By driving the arm cylinder SL2, the arm AR can be operated with respect to the boom BM. A bucket cylinder SL3 is attached between the arm AR and the bucket BK. By driving the bucket cylinder SL3, the bucket BK can be operated with respect to the arm AR. The above-described upper slewing body 32, boom BM, arm AR, and bucket BK provided in the working machine 3, which is a hydraulic excavator, are one aspect of the movable parts of the working machine 3. Also, the bucket BK is an example of the configuration of the "working tool" according to the present disclosure. The bucket BK is provided with a cutting edge BKT used for excavation and the like.

[0017] (Configuration of the driver's cab) FIG. 3 is a diagram showing the configuration of the driver's cab 32A of the working machine according to the first embodiment.

[0018] As shown in FIG. 3, the driver's cab 32A is provided with operation levers L1, L2, foot pedals F1, F2, and travel levers R1, R2. The operation levers L1, L2, foot pedals F1, F2, and travel levers R1, R2 are included in an operation unit 32AO for operating the movable parts of the working machine 3. The operation lever L1 and the operation lever L2 are arranged on the left and right sides of the seat ST in the driver's cab 32A. Also, the foot pedal F1 and the foot pedal F2 are arranged on the floor surface in front of the seat ST in the driver's cab 32A.

[0019] An example of an operation pattern showing the correspondence between the input operations for the operation levers L1, L2, travel levers R1, R2 and the operations of the working machine 3, which is a hydraulic excavator, is as follows.

[0020] The operation lever L1 arranged on the left side toward the front of the driver's cab is an operation mechanism for performing the slewing operation of the upper slewing body 32 and the digging / dumping operation of the arm AR. Specifically, when the operator of the working machine 3 tilts the operation lever L1 forward, the arm AR performs a dumping operation. Also, when the operator of the working machine 3 tilts the operation lever L1 backward, the arm AR performs a digging operation. Further, when the operator of the working machine 3 tilts the operation lever L1 to the right, the upper slewing body 32 slews to the right. Also, when the operator of the working machine 3 tilts the operation lever L1 to the left, the upper slewing body 32 slews to the left. When the operation lever L1 is tilted in the front-rear direction, the upper slewing body 32 may slew to the right or left, and when the operation lever L1 is tilted in the left-right direction, the arm AR may perform a dumping operation or a digging operation.

[0021] The operation lever L2 arranged on the right side facing the front of the driver's cab is an operation mechanism for performing the excavation / dumping operation of the bucket BK and the raising / lowering operation of the boom BM. Specifically, when the operator of the work machine 3 tilts the operation lever L2 forward, the lowering operation of the boom BM is executed. Also, when the operator of the work machine 3 tilts the operation lever L2 backward, the raising operation of the boom BM is executed. Further, when the operator of the work machine 3 tilts the operation lever L2 to the right, the dumping operation of the bucket BK is performed. Also, when the operator of the work machine 3 tilts the operation lever L2 to the left, the excavation operation of the bucket BK is performed.

[0022] Also, the travel levers R1, R2 are operation mechanisms for controlling the operation of the lower travel body 31, that is, for controlling the travel of the work machine 3. The travel lever R1 arranged on the left side facing the front of the driver's cab corresponds to the rotational drive of the left crawler CL of the lower travel body 31. Specifically, when the operator of the work machine 3 tilts the travel lever R1 forward, the left crawler CL rotates in the forward direction. Also, when the operator of the work machine 3 tilts the travel lever R1 backward, the left crawler CL rotates in the reverse direction.

[0023] The travel lever R2 arranged on the right side facing the front of the driver's cab corresponds to the rotational drive of the right crawler CR of the lower travel body 31. Specifically, when the operator of the work machine 3 tilts the travel lever R2 forward, the right crawler CR rotates in the forward direction. Also, when the operator of the work machine 3 tilts the travel lever R2 backward, the right crawler CR rotates in the reverse direction. Note that the foot pedals F1, F2 are interlocked with the travel levers R1, R2 respectively, and the travel can also be controlled by the foot pedals F1, F2.

[0024] Note that the above-described operation patterns are only examples and are not limited to the above aspects depending on the model of the hydraulic excavator and the like.

[0025] Note that depending on the embodiment, the work machine 3 described with reference to FIG. 2 may not be provided with the GNSS antennas G1, G2.

[0026] (Functional Configuration of the Reproduction System) FIG. 4 is a diagram showing the functional configuration of the playback system according to the first embodiment. Hereinafter, with reference to FIG. 4, the functions of the playback system 10 according to the first embodiment will be described. As shown in FIG. 4, the playback system 10 includes a CPU 100, a memory 101, a display unit 102, an operation reception unit 103, a communication interface 104, and a storage 105. Note that the CPU (Central Processing Unit) 100 may be a processor such as an FPGA or a GPU instead of a CPU.

[0027] The CPU 100 is a processor that controls the overall operation of the playback system 10. Various functions of the CPU 100 will be described later.

[0028] The memory 101 is a so-called main memory device. Instructions and data necessary for the CPU 100 to operate based on a program are expanded in the memory 101.

[0029] The display unit 102 is a display device capable of visibly displaying information, such as a liquid crystal display or an organic EL display.

[0030] The operation reception unit 103 is an input device, such as a general mouse, keyboard, touch sensor, etc.

[0031] The communication interface 104 is a communication interface for communicating with the data logger 20.

[0032] The storage 105 is a so-called auxiliary storage device, such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc. Log information TL received from the data logger 20, the vehicle type of the working machine 3, and the working machine model TM, which is a 3D model prepared in advance for each model, are recorded in the storage 105. Note that the working machine model TM will be described later. In addition, the storage 105 records a unit operation prediction model PM1, which is a learned machine learning model used when estimating the operation content of the working machine 3, an element operation prediction model PM2, heatmaps (H1, H2) generated during the estimation process, the estimated operation content R of the working machine 3, 3D point cloud information MAP representing the terrain information of the work site, and the like. Note that the unit operation prediction model PM1, the element operation prediction model PM2, and the heatmaps (H1, H2) will be described later. The 3D point cloud information MAP includes, for example, 3D point cloud information measured at the work site using a drone or the like, and information obtained by sequentially updating the terrain information from the cutting edge coordinate information of the working machine 3 during operation, and is assumed to be recorded in the storage 105.

[0033] The functions of the CPU 100 of the reproduction system 10 according to the first embodiment will be described in detail. By operating based on a predetermined program, the CPU 100 exhibits functions as an acquisition unit 1000, a reception unit 1001, an extraction unit 1002, an image generation unit 1003, an estimation unit 1004, a determination unit 1005, a synchronization unit 1006, and a display control unit 1007. Note that the above-mentioned predetermined program may be for realizing a part of the functions to be exhibited by the reproduction system 10. For example, the program may exhibit functions in combination with other programs already stored in the storage 105, or in combination with other programs implemented in other devices. In other embodiments, the reproduction system 10 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, part or all of the functions realized by the processor may be realized by the integrated circuit.

[0034] The acquisition unit 1000 acquires log information TL to be played back from among a plurality of log information TL recorded and stored in the storage 105. Here, it is assumed that the plurality of log information TL are recorded separately for each file recorded with different file names in the storage 105. The acquisition unit 1000 acquires, for example, one log information TL to be analyzed or two log information TL to be compared in accordance with an instruction from an operator. Hereinafter, the case where the acquisition unit 1000 acquires two log information TL to be compared will be described as an example. One of the two log information acquired by the acquisition unit 1000 is log information specified by the operator as a playback target by a file name or the like. Hereinafter, this log information will be denoted as specified log information TL1 (first log information). The other one of the two log information is log information that is played back simultaneously with the playback based on the specified log information TL1. Hereinafter, this log information will be denoted as comparison log information TL2 (second log information). The comparison log information TL2 may be log information that has been previously selected as a "model" for the work content for each of various work contents and conditions, for example, from the viewpoints of good fuel efficiency, short working hours, and driving by a skilled operator. Alternatively, the comparison log information TL2 may be, for example, past log information recorded in a previous drive performed by the same operator as the specified log information TL1. In this case, by comparing the two log information, for example, the proficiency of the operator can be analyzed.

[0035] The acquisition unit 1000 according to the present embodiment automatically searches for and acquires appropriate comparison log information TL2 as a comparison target for the specified log information TL1 based on the determination result by the determination unit 1005 described later. However, the acquisition unit 1000 according to other embodiments is not limited to the above aspect. For example, the acquisition unit 1000 may be an aspect in which the log information specified by the operator by a file name or the like is acquired as the comparison log information TL2. Further, when there is only one piece of recorded comparison log information TL2, the acquisition unit 1000 may be an aspect in which the one piece of comparison log information TL2 is acquired.

[0036] The reception unit 1001 receives a predetermined reproduction instruction from the operator of the reproduction system 10. For example, the reception unit 1001 receives a reproduction instruction of the working machine 3 from the operator of the reproduction system 10.

[0037] The extraction unit 1002 extracts the angle information used for reproducing the working machine 3 from the acquired log information TL. The angle information is an example of the "motion information" according to the present disclosure.

[0038] The image generation unit 1003 generates a motion image representing the operation of the working machine 3 by sequentially applying the motion information representing the operation of the working machine 3 based on the log information TL to the working machine model TM, which is a three-dimensional model of the working machine 3. As will be described later, in the present embodiment, the working machine model TM includes an operation unit 3D model M1 (an example of the operation unit model of the present disclosure) representing the operation unit 32AO provided in the cab 32A of the working machine 3. Further, the image generation unit 1003 generates a motion image by setting the viewpoint PT when projecting the working machine model TM onto a two-dimensional projection plane at a position facing the operation unit 32AO in the cab 32A. The image generation unit 1003 generates a motion image by setting the working machine model TM at a viewpoint facing the operation unit 32AO in the cab 32A.

[0039] The estimation unit 1004 estimates the work content of the working machine 3 at each time from the acquired log information TL.

[0040] The determination unit 1005 determines whether the comparison log information TL2 is appropriate as a comparison target for the designated log information TL1 based on the information included in the designated log information TL1.

[0041] The synchronization unit 1006 performs a process of synchronizing the reproduction of the working machine model TM based on the designated log information TL1 and the reproduction of the working machine model TM based on the comparison log information TL2. Specifically, the synchronization unit 1006 specifies the reproduction start time in the time series of the animation of the working machine model TM based on the work content estimated by the estimation unit 1004.

[0042] The display control unit 1007 causes the display unit 102 to display the operation image generated by the image generation unit 1003. Further, the display control unit 1007 may further cause the display unit 102 to display a work content image representing the time series of the estimated work content. Further, the display control unit 1007 may further cause the display unit 102 to display an operation amount image representing the time series of values corresponding to the operation amount of the operation unit 32AO included in the log information TL. Further, the display control unit 1007 may further cause the display unit 102 to display an instantaneous fuel consumption image representing the time series of the instantaneous fuel consumption included in the log information TL. Further, the display control unit 1007 may synthesize an image (topography image) representing the topography based on the three-dimensional point cloud information MAP with, for example, the operation image, and cause the display unit 102 to display the synthesized image.

[0043] (Processing Flow of the Reproduction System) Hereinafter, with reference to FIGS. 5 to 18, the specific processing flow performed by the reproduction system 10 will be described in detail.

[0044] The processing flow shown in FIG. 5 starts from the point in time when a dedicated application is launched by the operator of the reproduction system 10. When a dedicated application is launched by the operator's operation, the acquisition unit 1000 of the CPU 100 expands and acquires the designated log information TL1 designated as the reproduction target in the memory 101 (step S00).

[0045] Here, the log information TL (designated log information TL1, comparison log information TL2) will be described with reference to FIGS. 6 to 8.

[0046] As shown in FIGS. 6 to 8, the log information TL includes work machine identification information. Specifically, the work machine identification information is an individual identification number for individually identifying the work machine 3. In FIGS. 6 to 8, it is assumed that the work machine identification information is allocated so as to correspond to the vehicle type, model, type, machine number, etc. of the work machine 3 indicating a hydraulic excavator, a wheel loader, a bulldozer, etc. Note that the work machine identification information may be a number, alphabet, symbol, or a combination thereof, etc. other than a number.

[0047] As shown in FIG. 6, the log information TL includes information indicating the position and orientation of the working machine 3 at each time, and angle information of the movable parts of the working machine 3. Specifically, in the log information TL, the position of the working machine 3, the roll angle of the working machine 3 which is the inclination in the left - right direction of the machine body, the pitch angle which is the inclination in the front - rear direction of the machine body, the turning angle, the boom angle, the arm angle, and the bucket angle are recorded for each time. Here, the data logger 20 mounted on the working machine 3 is, for example, information obtained by receiving GNSS antennas G1 and G2, and based on the positioning information indicating latitude, longitude, and altitude, it identifies and records the position of the working machine 3. Further, the data logger 20 calculates and records the roll angle and pitch angle of the working machine 3 based on the measurement results of an IMU (Inertial Measurement Unit) mounted on the working machine 3. Also, the data logger 20 calculates and records the turning angle of the upper swing body 32 based on the positioning information obtained from each of the GNSS antennas G1 and G2 provided on the upper swing body 32. Furthermore, the data logger 20 calculates and records the boom angle, arm angle, and bucket angle based on the extension and contraction degrees of the boom cylinder SL1, arm cylinder SL2, and bucket cylinder SL3, respectively. Note that the boom angle, arm angle, bucket angle, and turning angle may be obtained, for example, by attaching IMUs to the boom, arm, bucket, and upper swing body and using those IMUs to acquire the boom angle, arm angle, bucket angle, and turning angle.

[0048] Note that the position, roll angle, and pitch angle are information necessary to identify the position and orientation of the working machine 3 itself. Therefore, for example, in an embodiment where only the movement of the movable parts of the working machine 3, that is, the upper swing body 32, boom BM, arm AR, and bucket BK, is animated and the position and orientation of the working machine 3 itself are not reproduced, the information on the position, roll angle, and pitch angle does not need to be included in the log information.

[0049] The log information TL shown in FIG. 6 corresponds to the "operation information" according to the present disclosure. The operation information is information representing the operations of movable parts of the working machine 3, such as the working machine 32B, the upper swing body 32, and the lower traveling body 31, which are operated by the operation unit 32AO of the working machine 3, for example. Alternatively, the log information TL shown in FIG. 6 is information representing the operations of the working machine 3 that can be visually recognized from the outside of the working machine 3, and can also be referred to as appearance operation information. That is, in the present embodiment, the operation information or the appearance operation information includes the position information, roll angle, pitch angle, swing angle, boom angle, arm angle, and bucket angle of the working machine 3. In the present embodiment, different from the drive mechanism information and operation information described later, the operation information (or appearance operation information) has the feature of being less affected by the difference in the vehicle class of the working machine 3 when estimating the work content.

[0050] Also, as shown in FIG. 7, the log information TL includes the degree of input to the operation levers L1, L2, etc. of the operator at each time, that is, the pilot hydraulic pressure (PPC (Proportional Pressure Control) pressure) indicating the degree of tilt of the lever and the degree of depression of the pedal. Specifically, the log information TL records the PPC pressures of the operation levers L1, L2, the travel levers R1, R2, or the foot pedals F1, F2 corresponding to each operation type of left / right swing, arm excavation / dump, boom raise / lower, bucket excavation / dump, right crawler forward / reverse, and left crawler forward / reverse by the operator at each time. Each time shown in FIG. 7 corresponds to each time in FIG. 6. In the present embodiment, each information shown in FIG. 7 is also referred to as operation information.

[0051] Further, as shown in FIG. 8, the log information TL includes information indicating the status of the main drive mechanisms such as the engine and hydraulic pump of the work machine 3 at each time. Specifically, in the log information TL, the engine coolant temperature, engine output, instantaneous fuel consumption, and oil temperature of the hydraulic pump are recorded for each time. Note that the items shown in FIG. 8 are just examples. For example, items such as the engine coolant temperature and the oil temperature of the hydraulic pump may be omitted, or for example, the fuel throttle opening may be included as other items affecting fuel consumption. Further, as yet another example, in an electrified construction machine, items such as instantaneous power consumption and battery remaining amount may be included. Also, each time shown in FIG. 8 corresponds to each time in FIGS. 6 and 7. In the present embodiment, each piece of information shown in FIG. 8 is also referred to as drive mechanism information.

[0052] Returning to FIG. 5, the estimation unit 1004 of the CPU 100 estimates the work content of the work machine 3 at each time based on the designated log information TL1 acquired in step S00 (step S01).

[0053] Here, the procedure for the estimation unit 1004 to estimate the work content of the work machine 3 from the log information TL will be described with reference to FIG. 9. The estimation unit 1004 estimates the work content of the work machine 3 with respect to both unit work and elemental work. Unit work is work that accomplishes one work objective. Elemental work is an element that constitutes unit work and indicates a series of operations or work classified by purpose.

[0054] Examples of the classification of unit work include, for example, "excavation and loading", "hoe work", "slope (from below)", "loading collection", "traveling", "stopping and parking" shown in FIG. 9, as well as "ditch excavation", "backfilling", "slope (from above)", and the like. Excavation and loading is work that digs, cuts, and loads the excavated earth and sand or rock onto the loading platform of a transport vehicle. Excavation and loading is a unit work composed of excavation, loading swing, soil discharge, empty swing, waiting for soil discharge, and load platform pressing. Scraping is an operation of scraping flat to make the extra undulations on the ground reach a predetermined height. Scraping consists of excavation and soil discharge, or excavation, load slewing, soil discharge, and empty slewing, and is a unit operation that may include leveling and brooming. Slope making (from below) is an operation of making a slope by the working machine 3 located below the target location. Slope making (from below) consists of compaction, excavation, load slewing, soil discharge, and empty slewing, and is a unit operation that may include leveling. Loading collection is an operation of collecting the earth and sand excavated by excavation or the like before loading it onto the transport vehicle. Loading collection consists of excavation, load slewing, soil discharge, and empty slewing, and is a unit operation that may include leveling. Traveling is an operation of moving the working machine 3. Traveling as a unit operation is a unit operation composed of traveling as an elemental operation. Stopping and idling is a state where there is no earth, sand, and rock in the bucket BK and the machine has stopped for a predetermined time or more. Stopping and idling as a unit operation is a unit operation composed of stopping as an elemental operation. Trench excavation is an operation of digging and scraping the ground into an elongated groove shape. Trench excavation consists of excavation, load slewing, soil discharge, and empty slewing, and is a unit operation that may include leveling. Backfilling is an operation of putting earth and sand into an existing groove or hole in the ground and filling it flat. Backfilling consists of excavation, load slewing, soil discharge, compaction, and empty slewing, and is a unit operation that may include leveling and brooming. Slope making (from above) is an operation of making a slope by the working machine 3 located above the target location. Slope making (from above) consists of compaction, excavation, load slewing, soil discharge, and empty slewing, and is a unit operation that may include leveling.

[0055] Examples of the classification of elemental operations include "excavation", "load slewing", "waiting for soil discharge", "soil discharge", "empty slewing", "suppressing the loading platform" shown in Fig. 9, and in addition, "compaction", "leveling", "brooming", etc. Excavation is an operation of digging and scraping earth and sand or rock with the bucket BK. The load swing is an operation of swinging the upper swing body 32 while holding the excavated earth and sand or rock in the bucket BK. The waiting for dumping is an operation of waiting for a transport vehicle for loading while holding the excavated earth and sand or rock in the bucket BK. The dumping is an operation of unloading the excavated earth and sand or rock from the bucket BK to a transport vehicle or a predetermined location. The empty load swing is an operation of swinging the upper swing body 32 with no earth and sand or rock in the bucket BK. The load bed pressing is an operation of pressing and flattening the earth and sand loaded on the load bed of the transport vehicle from above with the bucket BK. The rolling compaction is an operation of pushing the earth and sand into the disturbed ground with the bucket BK to form and strengthen the ground. The leveling is an operation of leveling the earth and sand with the bottom surface of the bucket BK. The fanning is an operation of leveling the earth and sand with the side surface of the bucket BK.

[0056] The estimation unit 1004 obtains a time series of likelihoods related to unit operations by inputting the log information TL into the unit operation prediction model PM1 in chronological order. The unit operation prediction model PM1 is a model that outputs the likelihood related to unit operations when the log information TL is input, for example, by learning using teacher data, and may be stored in the storage 105, for example.

[0057] Also, the estimation unit 1004 obtains a time series of likelihoods related to elemental operations by inputting the log information TL into the elemental operation prediction model PM2 in chronological order. The elemental operation prediction model PM2 is a model that outputs the likelihood related to elemental operations when the log information TL is input, for example, by learning using teacher data, and may be stored in the storage 105, for example.

[0058] The estimation unit 1004 smooths the time series of likelihoods related to unit operations and the time series of likelihoods related to elemental operations by applying them to a temporal average filter respectively, and generates a unit operation heatmap H1 representing the smoothed time series of likelihoods related to unit operations and an elemental operation heatmap H2 representing the smoothed time series of likelihoods related to elemental operations as shown in FIG. 9. The heatmaps H1 and H2 are maps in which, based on the smoothed time series of likelihoods, colors representing the likelihoods of operation categories are assigned to a plane with the operation categories on the vertical axis and time on the horizontal axis. The colors related to the heatmap may, for example, approach blue as the likelihood of the operation category is lower and approach red as the likelihood of the operation category is higher. The estimation unit 1004 stores the heatmaps H1 and H2 in the storage 105.

[0059] Based on the smoothed time series of likelihoods, the estimation unit 1004 identifies the time period in which the likelihood of the unit operation is dominant, and estimates the operation content of the work machine 3 during that time period. For example, for the time period in which the likelihood of the unit operation "excavation and loading" is dominant, the operation content of the work machine 3 is estimated to be "excavation and loading". Similarly, based on the smoothed time series of likelihoods, the estimation unit 1004 identifies the time period in which the likelihood of the elemental operation is dominant, and estimates the operation content of the work machine 3 during that time period. For example, for the time period in which the likelihood of the elemental operation "excavation" is dominant, the operation content of the work machine 3 is estimated to be "excavation". The estimation unit 1004 stores the information on the operation content R estimated for the work machine 3 in the storage 105.

[0060] Returning to FIG. 5, subsequently, acquisition unit 1000 determines whether to display information to be compared (comparison log information TL2) for the designated log information TL1 designated as the reproduction target based on a predetermined instruction operation from the operator (step S02A). If it is determined to display (step S02A: YES), after executing the processes of step S02B and step S03, the process proceeds to step S04. If it is not determined to display (step S02A: NO), the process proceeds to step S04 without executing step S02B and step S03. If it is determined to display (step S02A: YES), acquisition unit 1000 acquires appropriate comparison log information TL2 as the comparison target for the designated log information TL1 acquired in step S00 (step S02B). At this time, determination unit 1005 performs a determination processing flow for determining whether the plurality of comparison log information TL2 recorded in storage 105 is appropriate as the comparison target for the designated log information TL1 acquired in step S00. In this determination processing flow, first, determination unit 1005 selects one from among the plurality of comparison log information TL2 previously recorded in storage 105. Then, based on the log information TL, determination unit 1005 determines, for example, whether the vehicle compartment, the amount of change in the turning angle (the magnitude of the swing), the bucket height when the working machine 3 is waiting for dumping, etc. are of the same degree. If it is determined that they are not of the same degree, a new piece of comparison log information TL2 is selected, and the same determination process as above is performed. In addition, the determination unit 1005 according to another embodiment may narrow down the reproduction candidates based on the working position, working time, work classification, etc., in addition to the above determination process.

[0061] Returning to FIG. 5, next, synchronization unit 1006 performs a process of synchronizing the animation reproduction of the working machine model TM based on the designated log information TL1 and the animation reproduction of the working machine model TM based on the comparison log information TL2 (step S03). Specifically, synchronization unit 1006 identifies the reproduction start time in the time series in the animation of the working machine model TM according to the designated log information TL1 and the reproduction start time in the time series in the animation of the working machine model TM according to the comparison log information TL2.

[0062] The method for specifying the playback start time for synchronizing the two is as follows. That is, the synchronization unit 1006 extracts the timing of the switching of the unit operation in the designated log information TL1 based on the unit operation heat map H1 for the designated log information TL1 generated by the estimation unit 1004. Next, the synchronization unit 1006 extracts the timing of the switching of the unit operation in the comparison log information TL2 based on the unit operation heat map H1 for the comparison log information TL2 generated by the estimation unit 1004. The synchronization unit 1006 selects one of the extracted timings and specifies it as the playback start time of each animation. Note that the playback system 10 according to other embodiments is not limited to the above aspect. The playback system 10 according to other embodiments may, for example, compare the combination of angle information such as boom, arm, and bucket in the comparison log information TL2 with the combination of angle information such as boom, arm, and bucket in the designated log information TL1, extract the timing when both are at similar angles, and specify this as the playback start time of each animation.

[0063] After step S03, or when the answer is "NO" in step S02A, the reception unit 1001 receives a playback instruction from the operator (step S04). One aspect of the playback instruction may be an operation such as pressing a playback button. Also, the playback instruction may include information such as time, the position of the working machine 3, and various events such as an abnormality occurring in the working machine 3 as the starting point of playback.

[0064] Next, the acquisition unit 1000 selects and reads out the working machine model TM corresponding to the referenced working machine identification information from the storage 105 based on the working machine identification information as the type of the working machine 3 received by the reception unit 1001 (step S05).

[0065] Here, the working machine model TM will be described with reference to FIG. 10. As shown in FIG. 10, the construction machine model TM is information including construction machine identification information, an outer shape 3D model M0 of the construction machine 3 indicated by the construction machine identification information, and an operation unit 3D model M1 which is a three-dimensional model of the operation unit 32AO. The outer shape 3D model M0 is a 3D model representing the construction machine 3, and is constructed for each part of the construction machine 3 such as the lower traveling body and the upper slewing body. For example, the outer shape 3D model M0 represents the shape of the construction machine 3. For example, the outer shape 3D model M0 includes a lower traveling body outer shape model M01 representing the lower traveling body 31 of the construction machine 3, an upper slewing body outer shape model M02 representing the upper slewing body 32, a boom outer shape model M03 representing the boom BM, an arm outer shape model M04 representing the arm AR, and a bucket outer shape model M05 representing the bucket BK.

[0066] The operation unit 3D model M1 is a 3D model representing the operation unit 32AO, and represents the cab 32A provided with the operation unit 32AO, the seat ST in the cab 32A, and the three-dimensional shape representing the operation unit 32AO. Among these, models M11, M12, M13, M14, etc. representing the operation levers L1, L2, the traveling levers R1, R2, etc. included in the operation unit 32AO are models whose inclination changes based on the viewpoint PT when projecting the three-dimensional model onto the two-dimensional projection plane PS by the log information TL, or a straight line or plane passing through the viewpoint PT. The cab 32A, the seat ST, etc. do not change in inclination with respect to the viewpoint PT, or a straight line or plane passing through the viewpoint PT. In FIG. 10, the projection plane PS is a two-dimensional plane when projecting a three-dimensional solid shape onto a two-dimensional plane. The viewpoint PT is the position from which to view the three-dimensional solid shape projected onto the projection plane PS. In the present embodiment, the viewpoint PT is set at a position facing the operation unit 32AO inside the cab 32A. The viewpoint PT can be set, for example, at a position corresponding to the position of the eyes when the operator is sitting on the seat ST. The vector PTD represents the reference direction when viewing the projection plane PS from the viewpoint PT, for example, the front direction.

[0067] Returning to FIG. 5, the extraction unit 1002 extracts information for playback from each of the specified log information TL1 and the comparison log information TL2, or from the specified log information TL1 (step S06). For example, the extraction unit 1002 extracts various angle information such as boom angle, arm angle, bucket angle, etc. as information for playback, and the PPC pressures of each of the operation levers L1, L2, travel levers R1, R2, or foot pedals F1, F2. Also, in steps S00 and S02B, only the information for playback may be acquired. Hereinafter, the flow of the process will be described by taking as an example the case of comparing and playing back the specified log information TL1 and the comparison log information TL2. When playback using the comparison log information TL2 is not performed, it can be carried out by basically omitting the process related to the comparison log information TL2.

[0068] Next, the image generation unit 1003 and the display control unit 1007 execute simultaneous playback of the animation of the work machine model TM based on the specified log information TL1 and the animation playback of the work machine model TM based on the comparison log information TL2 (step S07). Here, the image generation unit 1003 applies the various information recorded in the specified log information TL1 to the work machine model TM in the order from the oldest time stamp from the playback start time specified in the synchronization process of step S03, and generates an operation image representing the operation of the work machine 3 (step S07a). Also, simultaneously with the generation of the operation image for this animation, the image generation unit 1003 applies the various information recorded in the comparison log information TL2 to the work machine model TM in the order from the oldest time stamp from the playback start time specified in the synchronization process of step S03, and generates an operation image representing the operation of the work machine 3 (step S07a). Also, in step S07a, the display control unit 1007 causes the display unit 102 to display the operation image generated by the image generation unit 1003.

[0069] Next, the content of the operation image generation process by the image generation unit 1003 in step S07a will be described in detail. The image generation unit 1003 changes the angles of the corresponding parts of the outer shape 3D model M0 based on various angle information such as the turning angle and boom angle shown in the log information TL (designated log information TL1, comparison log information TL2). For example, the image generation unit 1003 tilts the bucket outer shape model M05 around the rotation axis defined at the connection position with the arm outer shape model M04 to the bucket angle shown in the log information TL, thereby reproducing the position and posture of the bucket BK of the working machine 3.

[0070] Similarly, the image generation unit 1003 tilts the arm outer shape model M04 around the rotation axis defined at the connection position with the boom outer shape model M03 to the arm angle shown in the log information TL, thereby reproducing the position and posture of the arm AR of the working machine 3.

[0071] Similarly, the image generation unit 1003 tilts the upper swing body outer shape model M02 around the rotation axis defined at the connection position with the lower traveling body outer shape model M01 to the turning angle shown in the log information TL, thereby reproducing the position and posture of the upper swing body 32 of the working machine 3.

[0072] Similarly, the image generation unit 1003 tilts the lower traveling body outer shape model M01 around the roll rotation axis defined in the lower traveling body outer shape model M01 to the roll angle shown in the log information TL, and tilts it around the pitch rotation axis defined in the lower traveling body outer shape model M01 to the pitch angle shown in the log information TL, thereby reproducing the posture of the upper swing body 32 of the working machine 3.

[0073] In addition, the reproduction system 10 according to the first embodiment can reproduce the traveling animation of the working machine 3 based on the PPC pressures of the right crawler forward / backward and the left crawler forward / backward at each time included in the log information TL.

[0074] Specifically, based on the PPC pressures of the right crawler forward / backward and the left crawler forward / backward, the outer shape 3D model M0 is moved forward, backward, left and right forward, and left and right backward. For example, based on the numerical values of the PPC pressures of the right crawler forward and the left crawler forward, the outer shape 3D model M0 is moved in the forward direction. The moving speed may be changed based on the numerical values of the PPC pressures.

[0075] Also, based on the numerical values of the PPC pressures of the right crawler backward and the left crawler backward, the outer shape 3D model M0 is moved in the backward direction. Also, based on the difference in the numerical values of the PPC pressures of the right crawler forward and the left crawler forward, the outer shape 3D model M0 is moved to curve in the left and right forward direction. For example, when the numerical value of the PPC pressure of the right crawler forward is greater than the numerical value of the PPC pressure of the left crawler forward, it is moved to curve in the forward left direction. The moving speed and the curvature size may be changed according to the respective numerical values of the PPC pressures and the difference in the numerical values of the PPC pressures.

[0076] Similarly, based on the difference in the numerical values of the PPC pressures of the right crawler backward and the left crawler backward, the outer shape 3D model M0 is moved to curve in the left and right backward direction. For example, when the numerical value of the PPC pressure of the right crawler backward is greater than the numerical value of the PPC pressure of the left crawler backward, it is moved to curve in the backward left direction. The moving speed and the curvature size may be changed according to the respective numerical values of the PPC pressures and the difference in the numerical values of the PPC pressures.

[0077] In addition to the PPC pressures of the right crawler forward / backward and the left crawler forward / backward, by using the position information for playback, the running of the working machine 3 can be animated more accurately. In this case, by using the position information, the moving speed and position of the working machine 3 can be expressed more accurately. Also, in addition to the PPC pressures of the right crawler forward / backward and the left crawler forward / backward, by animating the working machine 3 based on the roll angle, or the pitch angle, or both the roll angle and the pitch angle, the left and right inclination of the working machine 3 during running or the front and back inclination of the working machine 3 can be reproduced.

[0078] In addition, the image generation unit 1003 changes the angle of the corresponding part of the outer shape 3D model M0 based on the PPC pressure shown in the log information TL (designated log information TL1, comparison log information TL2). For example, the image generation unit 1003 tilts the bucket outer shape model M05 around the rotation axis defined at the connection position with the arm outer shape model M04 to the bucket angle shown in the log information TL, thereby reproducing the position and posture of the bucket BK of the working machine 3.

[0079] Similarly, the image generation unit 1003 tilts the arm outer shape model M04 around the rotation axis defined at the connection position with the boom outer shape model M03 to the arm angle shown in the log information TL, thereby reproducing the position and posture of the arm AR of the working machine 3.

[0080] Similarly, the image generation unit 1003 tilts the upper swing body outer shape model M02 around the rotation axis defined at the connection position with the lower traveling body outer shape model M01 to the swing angle shown in the log information TL, thereby reproducing the position and posture of the upper swing body 32 of the working machine 3.

[0081] Similarly, the image generation unit 1003 tilts the lower traveling body outer shape model M01 around the roll rotation axis defined in the lower traveling body outer shape model M01 to the roll angle shown in the log information TL, and tilts it around the pitch rotation axis defined in the lower traveling body outer shape model M01 to the pitch angle shown in the log information TL, thereby reproducing the posture of the upper swing body 32 of the working machine 3.

[0082] Similarly, the image generation unit 1003 changes the inclination of the models M11 representing the operation lever L1, M12 representing the operation lever L2, M13 representing the traveling lever R1, and M14 representing the traveling lever R2 based on the PPC pressure.

[0083] In addition, as shown in FIG. 11, the image generation unit 1003 may rotate the angle of the projection plane PS in the direction of arrow RA based on the roll angle, and change the position (height) of the projection plane PS based on the pitch angle. That is, for example, as shown in FIG. 12, based on the region DS on the display screen of the display unit 102, the angular relationship with the projection plane PS may be changed like the planes PS1, PS2, etc. based on the roll angle. Alternatively, for example, as shown in FIG. 13, based on the region DS on the display screen of the display unit 102, the positional (height) relationship with the projection plane PS may be changed like the planes PS3, PS4, etc. based on the pitch angle. In this way, by changing the angle and position of the projection plane PS based on the roll angle and pitch angle, the changes in the roll angle and pitch angle, which are the changes in the posture of the working machine 3 as viewed from the outside, can be reflected in the two-dimensional image generated using the viewpoint PT inside the driver's seat 32A of the working machine 3. In this case, for example, in the event of vibrations or the like such that the roll angle and pitch angle of the working machine 3 change in response to an operation of the operation unit 32AO, this can be reflected as a change in, for example, the operation images, terrain images, etc. representing the models M11 to M14. Note that the image generation unit 1003 may be configured to be able to select whether to use, as the roll angle and pitch angle used when changing the angle and position of the projection plane PS, values based on the specified log information TL1 or values based on the comparison log information TL2, for example, according to a predetermined operation of the operator during image reproduction. According to this configuration, for example, by reproducing the image while switching the selection, the degree of occurrence of vibrations or the like can be compared.

[0084] Also, in step S07, the display control unit 1007 synthesizes a terrain image based on the three-dimensional point cloud information MAP with the operation image and causes the display unit 102 to display it (step S07b).

[0085] Also, in step S07, the display control unit 1007 may cause the display unit 102 to display a supplementary image representing the time series of various information including the estimation result of the work content and the log information TL (step S07c).

[0086] During the animation playback of the operations of the two working machines 3, the image generation unit 1003 determines whether to end the animation playback (step S08). For example, when receiving an instruction to end playback based on pressing a stop button or the like, it is determined to end the animation playback. After the start of the animation playback, it may be determined to end the animation playback after a predetermined period has elapsed. If the animation playback does not end (step S08; NO), the image generation unit 1003 continues the simultaneous animation playback of the two working machine models TM. On the other hand, when ending the animation playback (step S08; YES), the image generation unit 1003 ends the animation playback process.

[0087] Among the processing flows described with reference to FIG. 5, steps S00, S01, S03, S04, S05, S06, and S08 are not essential components of the playback system 10, and in other embodiments, such steps may not be provided.

[0088] (Display Image of the Playback System) Figures 14 to 18 show examples of display images of the display unit 102 according to the first embodiment. Figures 14 to 17 show examples of display images based on the designated log information TL1 and the comparison log information TL2 of the playback system according to the first embodiment. Figure 18 shows an example of a display image based on the designated log information TL1 of the playback system according to the first embodiment. The display image D2 shown in Figure 15 includes the operation image D21 in a state where a certain time has elapsed from the operation image D11 included in the display image D1 shown in Figure 14. The operation images D11 and D21 include images representing models M11, M12, M03, etc., and a terrain image D13 is synthesized. In the operation example shown in Figure 5, the operation images D11 and D21 are displayed in step S7a, and the terrain image D13 is displayed in step S7b. Also, buttons B1 and B2 for selecting the reference log information TL are displayed on the operation images D11 and D21. Figure 14 shows a state where button B1 is selected. In this case, the roll angle and pitch angle are set based on the designated log information TL1, and an image is generated. Figure 15 shows a state where button B2 is selected. In this case, the roll angle and pitch angle are set based on the comparison log information TL2, and an image is generated. In the examples shown in Figures 14 and 15, the changes in the roll angle and pitch angle are represented, for example, as changes in the inclination or position of the terrain image D13. By switching the selection states of buttons B1 and B2, it is possible to compare the state when the designated log information TL1 was recorded with the state when the comparison log information TL2 was recorded. Note that the image shown by the solid line is an image based on the designated log information TL1. The image shown by the dashed line is an image based on the comparison log information TL2.

[0089] Also, the display images D1 and D2 include the supplementary images D12 and D22 generated in step 7c. Figure 16 shows the supplementary image D12 shown in Figure 14. As shown in Figure 16, the supplementary image D12 includes a work content image D121 representing the time series of the work content estimated based on the designated log information TL1, a work content image D122 representing the time series of the work content estimated based on the comparison log information TL2, a playback time icon D129, and a scroll bar D120.

[0090] Further, the supplementary image D12 includes an operation amount image D123 and an operation amount image D124 that represent a time series of values corresponding to the operation amount of the operation unit 32AO included in the log information TL. Note that the solid line represents the operation amount based on the specified log information TL1, and the dashed line represents the operation amount based on the comparison log information TL2. Further, the operation amount image D123 represents the boom raising lever pressure. Further, the operation amount image D124 represents the arm excavation lever pressure.

[0091] FIG. 17 shows an example of the supplementary image D12 when the scroll bar D120 shown in FIG. 16 is scrolled downward. The supplementary image D12 shown in FIG. 18 includes operation amount images D125, D126, and D128, and an instantaneous fuel consumption image D127 that represents a time series of the instantaneous fuel consumption included in the log information TL. The operation amount image D125 represents the arm excavation lever pressure. The operation amount image D126 represents the boom raising lever pressure. The operation amount image D128 represents the bucket dump lever pressure. Note that the content of the time series information included in the supplementary image D12 shown in FIGS. 16 and 17 is an example and is not limited to this example. For example, some of the items shown in FIGS. 16 and 17 may be omitted.

[0092] Further, the display image D1A shown in FIG. 18 includes an operation image D11 and a supplementary image D12 based on the specified log information TL1.

[0093] (Function, effect) As described above, the playback system 10 according to the first embodiment includes an acquisition unit 1000 that acquires the log information TL of the working machine 3 associated with time, and an operation information representing the operation of the working machine 3 based on the log information TL is sequentially applied to the working machine model TM, which is a three-dimensional model of the working machine 3, to generate an operation image representing the operation of the working machine 3. An image generation unit 1003 and a display control unit 1007 that causes the operation image to be displayed on the display unit 102. The working machine model TM includes an operation unit 3D model M1 representing the operation unit 32AO provided in the cab 32A of the working machine 3. Further, the image generation unit 1003 generates an operation image by setting the working machine model TM at a viewpoint facing the operation unit 32AO in the cab 32A. According to this configuration, the movement of the operation unit 32AO can be reproduced, for example, based on the viewpoint of the operator. Therefore, according to the present embodiment, it is possible to easily grasp and reproduce the movement of the working machine 3.

[0094] Further, in the playback system 10 according to the first embodiment, since the operation image includes an image representing the operation unit 32AO, for example, lever operations can be easily analyzed and compared. Further, the playback system 10 can display on the display unit 102 a supplementary image including an operation content image representing the time series of the operation content, an operation amount image representing the time series of the operation amount of the operation unit, an instantaneous fuel consumption image representing the time series of the instantaneous fuel consumption, etc., together with the operation image. According to this configuration, for example, for each work item, the state of the temporal change of the operation of the operation unit 32AO, etc., can be grasped in more detail compared to the case without the supplementary image. Examples of the state of the temporal change include, for example, whether the lever operation, etc., includes operations of intermediate operation amounts for fine adjustment in addition to fully open and fully closed, and in what flow each lever is operated for each operation item.

[0095] (Modification example) The content of the log information TL according to the first embodiment (Figs. 6 to 8) is not limited to this in other embodiments. For example, when the working machine 3 is not a hydraulic excavator but another vehicle type, the log information TL corresponding to the vehicle type is recorded. Other vehicle types are, for example, wheel loaders, bulldozers, etc.

[0096] Further, although the playback system 10 according to the first embodiment is described as being installed at a location separated from the working machine 3 and connected to the data logger 20 mounted on the working machine 3 via a wide area communication network, it is not limited to this mode in other embodiments.

[0097] For example, in the playback system 10 according to other embodiments, part or all of the configuration of the playback system 10 may be installed inside the working machine 3. In this case, the data logger 20 may transmit the log information TL to the playback system 10 via a network inside the working machine 3 or the like without using a wide area communication network. By doing so, the operator boarding the working machine 3 can animate and check the movement of the working machine 3 that the operator himself / herself operates on the spot. Also, by playing back the movement of the working machine 3 as a model for the operator of the working machine 3, it can be used as guidance.

[0098] Note that the playback system 10 installed inside the working machine 3 may acquire the log information TL of other working machines 3 via a wide area communication network or the like. By doing so, the state of the working machine 3 other than the working machine 3 on which the playback system 10 is mounted can be animated and played back.

[0099] Further, the playback system 10 according to other embodiments may be installed at a location separated from the working machine 3, and may be configured to transmit the video information generated by the animation playback process to the monitor mounted on the working machine 3 for display.

[0100] In another embodiment, as one aspect of the reproduction instruction received from the operator, for example, it may be to specify the reproduction period. For example, the reproduction period may be the reproduction start time and the reproduction end time. In this case, the reproduction system 10 reproduces the working machine 3 for the received reproduction period. In another embodiment, the specification of the reproduction end time is not essential. For example, in another embodiment, as the reproduction instruction from the operator, it may be a mode of receiving only the reproduction start time and reproducing for a certain period from the reproduction start time, or a mode of continuing to reproduce as long as the log information exists, or a mode of stopping reproduction in accordance with the occurrence of various other events.

[0101] Note that the log information TL (Figs. 6 to 8) to be acquired does not have to be arranged in chronological order. In this case, the image generation unit 1003 may apply the information used for reproduction among the log information TL to the working machine model TM in chronological order.

[0102] Note that the processes of the various processes of the reproduction system 10 described above are stored in a computer-readable recording medium in the form of a program, and the computer reads and executes this program to perform the above various processes. The computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, etc. Further, this computer program may be distributed to the computer via a communication line, and the computer that has received this distribution may execute the program.

[0103] The above program may be for realizing a part of the above-described functions. Further, it may be a so-called difference file, a difference program, etc., which can realize the above-described functions in combination with a program already recorded in the computer system.

[0104] Some or all of the functions of the above-described playback system 10 may be assigned to the working machine 3. For example, some or all of the functions of the acquisition unit 1000, reception unit 1001, extraction unit 1002, image generation unit 1003, estimation unit 1004, determination unit 1005, synchronization unit 1006, display control unit 1007, memory 101, display unit 102, operation reception unit 103, communication interface 104, and storage 105 may be assigned to the working machine 3.

[0105] As described above, some embodiments of the present disclosure have been described. However, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, as well as in the invention described in the claims and the equivalent scope thereof.

[0106] (Appendix) The playback system 10 according to the present disclosure is understood as follows, for example.

[0107] (1) The playback system 10 according to the first aspect includes an acquisition unit 1000 that acquires log information TL of the working machine 3 associated with time, and an image generation unit 1003 that generates an operation image representing the operation of the working machine by sequentially applying operation information representing the operation of the working machine based on the log information to a three-dimensional model (working machine model TM) of the working machine, and a display control unit 1007 that causes the operation image to be displayed on a predetermined display unit 102. The three-dimensional model includes an operation unit model (operation unit 3D model M1) representing an operation unit 32AO provided in the cab 32A of the working machine. The image generation unit generates the operation image by setting a viewpoint PT at a position facing the operation unit in the cab when projecting the three-dimensional model onto a two-dimensional projection plane PS. According to this aspect and the following aspects, it is possible to easily grasp and reproduce the movement of the working machine 3.

[0108] (2) The playback system 10 according to the second aspect is the playback system of (1), and the operation image includes an image representing the operation unit.

[0109] (3) The playback system 10 according to the third aspect is the playback system of (1) or (2), and further includes an estimation unit 1004 that estimates the work content of the work machine for each time based on the log information. The display control unit further displays, on the display unit, a work content image D121 representing the time series of the estimated work content.

[0110] (4) The playback system 10 according to the fourth aspect is the playback system of (1) to (3), and the display control unit further displays, on the display unit, an operation amount image representing the time series of values corresponding to the operation amounts of the operation unit included in the log information.

[0111] (5) The playback system 10 according to the fifth aspect is the playback system of (1) to (4), and the display control unit further displays, on the display unit, an instantaneous fuel consumption image representing the time series of the instantaneous fuel consumption included in the log information.

[0112] (6) The playback system 10 according to the sixth aspect is the playback system of (1) to (5), and the image generation unit rotates the projection plane based on the roll angle of the work machine and changes the height of the projection plane based on the pitch angle of the work machine.

Explanation of Reference Numerals

[0113] 1 Analysis Support System, 10 Playback System, 100 CPU, 1000 Acquisition Unit, 1001 Reception Unit, 1002 Extraction Unit, 1003 Image Generation Unit, 1004 Estimation Unit, 1005 Judgment Unit, 1006 Synchronization Unit, 1007 Display Control Unit, 101 Memory, 102 Display Unit, 103 Operation Reception Unit, 104 Communication Interface, 105 Storage, 20 Data Logger, 3 Work Machine, H1~H2 Heat Map, TM Work Machine Model, M1 Operation Unit 3D Model, TL Log Information, PM1 Unit Work Prediction Model, PM2 Element Work Prediction Model, R Estimated Work Content, D11, D21, D31 Operation Image

Claims

1. An acquisition unit that acquires log information of a working machine associated with a time; An image generation unit that generates an operation image representing the operation of the working machine by sequentially applying operation information representing the operation of the working machine based on the log information to a three-dimensional model of the working machine; A display control unit that causes the operation image to be displayed on a predetermined display unit and comprising: The three-dimensional model includes an operation unit model representing an operation unit provided in the cab of the working machine; The image generation unit generates the operation image by setting the three-dimensional model at a viewpoint facing the operation unit in the cab. A playback system.

2. The operation image includes an image representing the operation unit. The playback system according to claim 1.

3. The system further comprises an estimation unit that estimates the work content of the working machine for each time based on the log information; The display control unit further causes the display unit to display a work content image representing the time series of the estimated work content. The playback system according to claim 2.

4. The display control unit further causes the display unit to display an operation amount image representing the time series of values corresponding to the operation amount of the operation unit included in the log information. The playback system according to claim 3.

5. The display control unit further causes the display unit to display an instantaneous fuel consumption image representing the time series of the instantaneous fuel consumption included in the log information. The playback system according to claim 4.

6. The image generation unit rotates a projection plane on which the three-dimensional model is projected based on the roll angle of the working machine, and changes the height of the projection plane based on the pitch angle of the working machine. The playback system according to claim 5.

7. A step of acquiring log information of a working machine associated with a time; A step of generating an operation image representing the operation of the working machine by sequentially applying operation information representing the operation of the working machine based on the log information to a three-dimensional model of the working machine; A step of causing the operation image to be displayed on a predetermined display unit and including: The three-dimensional model includes an operation unit model representing an operation unit provided in the cab of the working machine; When generating the operation image, the operation image is generated by setting the three-dimensional model at a viewpoint facing the operation unit in the cab. A playback method.

Images