Working machine operating state calculation device

Abstract

To provide a device capable of easily learning an operating state that damages a work machine.SOLUTION: An operating state calculation device for a work machine includes a processor and an output device. The processor calculates a feature vector with operation information as a feature amount in each time, and assigns a classification ID to the feature vector on the basis of a classification model. The processor calculates damage received by the work machine at each time on the basis of a plurality of pieces of operation information. The processor stores a database obtained by associating time, the classification ID, and the damage with each other. The processor calculates a representative value of the damage of each classification ID on the basis of the database, generates a sentence representing an operating state of a work machine corresponding to the classification ID by connecting word phrases corresponding to the magnitude of a plurality of feature amounts constituting a representative feature vector of the classification ID, and outputs to the output device operation information obtained by associating the sentence with the representative value of the damage calculated for the classification ID linked to the representative feature vector used to generate the sentence.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a working machine motion state calculation device. [Background technology]

[0002] Patent Document 1 discloses an invention that visualizes the start and end times of stress amplitude by linking them to the work content of a work machine (empty load movement, earth loading, loaded load movement, etc.). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2015 / 132838 Summary of the Invention [Problem to be solved by the invention]

[0004] A work machine performs various operations during work, and its operating conditions, such as its posture and operating speed, change over time. The invention described in Patent Document 1 makes it possible to know the details of work, such as moving an empty load or loading earth and sand, but it is not possible to know specifically what operating conditions cause damage to the work machine, and in this respect there is room for improvement.

[0005] An object of the present invention is to provide a working machine motion state calculation device that can easily determine motion states that cause damage to the working machine. [Means for solving the problem]

[0006] A work machine operational state calculation device according to one aspect of the present invention comprises a processing device that calculates the operational state of the work machine based on multiple pieces of operational information accompanying the operation of the work machine, which change over time, and an output device that outputs operation information representing the operational state of the work machine calculated by the processing device. The processing device calculates a feature vector using the multiple pieces of operational information as feature quantities for each of multiple times, assigns a classification ID to the feature vector calculated for each of the multiple times based on a classification model in which a representative feature vector, which is a representative value of a cluster of feature vectors representing the operational state of the work machine, is associated with the classification ID, calculates damage suffered by the work machine for each of the multiple times based on the multiple pieces of operation information, stores a database that associates time with the classification ID assigned to the calculated feature vector and the calculated damage, and calculates a representative value of the damage for each classification ID based on the time, the classification ID, and the damage contained in the database. The plurality of classification IDs are classified into a plurality of groups, and based on the difference between the representative feature vectors linked to the plurality of classification IDs classified into the same group, a feature amount to be used for generating a sentence is identified, and the feature amount identified for each of the plurality of groups to which the classification ID belongs is By connecting words corresponding to the magnitude of the feature, a sentence representing the operating state of the work machine corresponding to the classification ID is generated, and operation information associating the sentence representing the operating state of the work machine with the representative value of damage calculated for the classification ID linked to the representative feature vector used to generate the sentence is output to the output device, and the output device outputs the operation information input from the processing device. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a working machine motion state calculation device that can easily determine motion states that cause damage to the working machine. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing the configuration of an operating state calculation device according to the first embodiment. [Figure 2] FIG. 2 is a functional block diagram of the motion state calculation device according to the first embodiment. [Figure 3A] FIG. 3A is a diagram illustrating an example of a method for generating a classification model. [Figure 3B] FIG. 3B is a diagram illustrating a specific example of the classification process performed by the classification processing unit. [Figure 4] FIG. 4 is a diagram for explaining an example of damage calculation processing by the damage calculation processing unit, and shows an example of time-series changes in stress, stress amplitude, and fatigue damage rate. [Figure 5] FIG. 5 is a diagram for explaining an example of the linking process by the linking processing unit, and shows a classification ID time-series table and a damage time-series table. [Figure 6] FIG. 6 is a diagram for explaining an example of statistical processing of damage by the statistical processing unit, and shows an example of time-series changes in classification ID, fatigue damage rate, and cumulative damage. [Figure 7] FIG. 7 is a graph showing the relationship between the classification ID and the amount of damage. [Figure 8] FIG. 8 is a flowchart showing an example of the flow of processing executed by the annotation generating unit. [Figure 9] FIG. 9 is a diagram illustrating the hierarchical clustering process. [Figure 10] FIG. 10 is a tree diagram illustrating an example of a process for selecting words and phrases as branching factors. [Figure 11] FIG. 11 shows an example of a display image in which the annotation text and the amount of damage are associated with each other, and shows a bar graph. [Figure 12] FIG. 12 is an example of a display image in which annotations and damage amounts are associated, showing a tree diagram in which the thickness of the branches varies depending on the amount of damage. [Figure 13] FIG. 13 is an example of a display image in which annotations and damage amounts are associated with each other, showing a tree diagram in which the line types of the branches vary depending on the amount of damage. [Figure 14] FIG. 14 is a functional block diagram of the motion state calculation device according to the second embodiment. [Figure 15] FIG. 15 is a functional block diagram of the motion state calculation device according to the third embodiment. [Figure 16] FIG. 16 is a functional block diagram of the motion state calculation device according to the fourth embodiment. [Figure 17] FIG. 17 is a diagram showing an example of information output from a display device and a speaker provided in the operator's cab of a hydraulic excavator. [Figure 18] FIG. 18 is a diagram showing an example of information output from a display device and a speaker provided in the cab of a dump truck. [Figure 19] FIG. 19 is a diagram showing an example of a display image of annotations and damage amounts hierarchically displayed by cataloging. [Figure 20] FIG. 20 is a functional block diagram of the motion state calculation device according to the second modification. DETAILED DESCRIPTION OF THE INVENTION

[0009] A working machine motion state calculation device according to an embodiment of the present invention will be described with reference to the drawings.

[0010] First Embodiment A construction machine motion state calculation device 1 according to a first embodiment of the present invention will be described with reference to Figures 1 to 13. Figure 1 is a diagram showing the configuration of the construction machine motion state calculation device 1. The motion state calculation device 1 includes a machine control device 110 provided on a construction machine 101 such as a hydraulic excavator 101A, a dump truck 101B, or a work robot 101C that performs work at a work site, various devices connected to the machine control device 110, a management server 150 provided in a management facility located away from the work site, and various devices connected to the management server 150.

[0011] The hydraulic excavator 101A is a work machine 101 that includes a traveling body 2, a rotating body 3 that is rotatably mounted on the traveling body 2, and an articulated front working implement (hereinafter simply referred to as the working implement) 4 that is attached to the rotating body 3. The hydraulic excavator 101A has a vehicle body that is made up of the traveling body 2 and the rotating body 3. The working implement 4 has a boom 11, an arm 12, a bucket 13, and hydraulic cylinders that drive these (boom cylinder 11a, arm cylinder 12a, and bucket cylinder 13a).

[0012] The dump truck 101B is a work machine 101 equipped with a running body (vehicle body), a loading platform (vessel) mounted on the running body on which cargo is loaded, and a working device having a hydraulic cylinder (hoist cylinder) for rotating (raising and lowering) the loading platform in the vertical direction. The work robot 101C is a work machine 101 equipped with a working device having an articulated arm and a gripper for gripping an object.

[0013] The management device 105 is made up of the management server 150 and various devices connected to the management server 150. The management device 105 is an external device that remotely manages the operating status of the work machine 101. Examples of management facilities include facilities such as the head office, branch office, and factory of the manufacturer of the work machine 101, a rental company for the work machine 101, a data center that specializes in operating servers, and a facility of the owner of the work machine 101.

[0014] The operating state calculation device 1 can calculate the operating state of various work machines such as a hydraulic excavator 101A, a dump truck 101B, and a work robot 101C, but below we will explain an example where the operating state calculation device 1 calculates the operating state of a hydraulic excavator 101A.

[0015] The hydraulic excavator 101A performing work at the work site and the management device 105 installed at a location remote from the work site are configured to be able to perform two-way communication via a wide area network communication line 109. In other words, the hydraulic excavator 101A and the management device 105 can send and receive information (data) via the communication line 109. The communication line 109 is a mobile phone communication network (mobile communication network) deployed by a mobile phone carrier or the like, the Internet, or the like.

[0016] The hydraulic excavator 101A is equipped with a machine control device 110 which is a control device that controls each part of the hydraulic excavator 101A, an input device 120 which inputs information to the machine control device 110 in response to operation by an operator, a plurality of detection devices 121 to 124 which detect operating information associated with the operation of the hydraulic excavator 101A, a display device 125 which displays images based on control signals from the machine control device 110, a speaker 126 which outputs sound based on control signals from the machine control device 110, and a communication device 128 which exchanges information with the management device 105.

[0017] The communication device 128 is a wireless communication device capable of wireless communication with the wireless base station 108 connected to the communication line 109, and has a communication interface including a communication antenna having a sensitivity band of, for example, the 2.1 GHz band. The communication device 128 exchanges information with the management server 150 and the like via the wireless base station 108 and the communication line 109.

[0018] The input device 120 is a switch device having a plurality of switches, a microphone, etc. The display device 125 is, for example, a liquid crystal display device or an organic EL display device. The display device 125 is an output device that outputs information as an image, and the speaker 126 is an output device that outputs information as sound.

[0019] The detection device 121 is a device that detects the pressure of the hydraulic cylinder, and includes a bottom pressure sensor that detects the pressure in the bottom-side oil chamber of the boom cylinder 11a, a rod pressure sensor that detects the pressure in the rod-side oil chamber of the boom cylinder 11a, a bottom pressure sensor that detects the pressure in the bottom-side oil chamber of the arm cylinder 12a, a rod pressure sensor that detects the pressure in the rod-side oil chamber of the arm cylinder 12a, a bottom pressure sensor that detects the pressure in the bottom-side oil chamber of the bucket cylinder 13a, and a rod pressure sensor that detects the pressure in the rod-side oil chamber of the bucket cylinder 13a.

[0020] The detection device 122 is a device that detects the attitude of the work implement 4, and calculates the boom angle, arm angle, bucket angle, and vehicle body tilt angle based on multiple angle sensors and signals from the multiple angle sensors, and outputs the calculation results to the machine control device 110. In this embodiment, the boom angle, arm angle, bucket angle, and vehicle body tilt angle are angles of the boom 11, arm 12, bucket 13, and vehicle body relative to a reference plane (horizontal plane). Note that the boom angle may also be expressed as the relative angle of the boom 11 with respect to the vehicle body, the arm angle as the relative angle of the arm 12 with respect to the boom 11, and the bucket angle as the relative angle of the bucket 13 with respect to the arm 12.

[0021] The detection device 123 includes a motor pressure sensor that detects the pressure of a swing motor that swings the swing unit 3 relative to the traveling unit 2. The detection device 124 includes a plurality of strain sensors (strain gauges) that detect strain at predetermined positions of the hydraulic excavator 101A. In addition, although not shown, the hydraulic excavator 101A is equipped with a plurality of detection devices that detect a plurality of pieces of operation information associated with the operation of the hydraulic excavator 101A.

[0022] The management device 105 includes a management server 150, an input device 161 that inputs information to the management server 150 in response to operations by the administrator, a communication device 162 that exchanges information with the hydraulic excavator 101A, and a display device 165 that displays images based on control signals from the management server 150. The display device 165 is, for example, a liquid crystal display device or an organic EL display device. The display device 165 is an output device that outputs information as an image.

[0023] The machine control device 110 and management server 150 are configured as computers equipped with CPUs (Central Processing Units) 111, 151 as operating circuits, ROMs (Read Only Memory) 112, 152 as storage devices, RAMs (Random Access Memory) 113, 153 as storage devices, input interfaces 114, 154, output interfaces 115, 155, and other peripheral circuits. The management server 150 and the machine control device 110 may each be configured as a single computer or multiple computers. The above computer configuration is an example, and operating circuits such as MPUs (Micro Processing Units), DSPs (Digital Signal Processors), ASICs (Application Specific Integrated Circuits), and FPGAs (Field Programmable Gate Arrays) may be used instead of the CPUs 111, 151.

[0024] The ROMs 112 and 152 are nonvolatile memories such as EEPROMs, and store programs capable of executing various calculations. In other words, the ROMs 112 and 152 are storage media from which programs for implementing the functions of this embodiment can be read. The RAMs 113 and 153 are volatile memories, and serve as work memories for directly inputting and outputting data to and from the CPUs 111 and 151. The RAMs 113 and 153 temporarily store necessary data while the CPUs 111 and 151 are executing the programs. The management server 150 and the machine control device 110 further include HDDs (Hard Disk Drives) 116 and 156 as storage devices. The HDDs 116 and 156 are nonvolatile large-capacity storage devices that store various information (data). While an example of storing various data in the HDDs 116 and 156 will be described below, various data can also be stored in various storage devices, such as SSDs (Solid State Drives) and flash memory, instead of the HDDs 116 and 156.

[0025] The CPUs 111, 151 are arithmetic devices that load a control program stored in the ROMs 112, 152 into the RAMs 113, 153 and execute the program, and perform predetermined arithmetic processing on information (data) received from the input interfaces 114, 154, the ROMs 112, 152, the RAMs 113, 153, the HDDs 115, 156, etc. in accordance with the control program. Signals are input to the input interfaces 114, 154 from various devices. The input interfaces 114, 154 convert the input signals into data that can be calculated by the CPUs 111, 151. The output interfaces 115, 155 generate output signals according to the results of calculations performed by the CPUs 111, 151, and output the signals to various devices.

[0026] 2 is a functional block diagram of the operational state calculation device 1 according to the first embodiment of the present invention. The management server 150 acquires a plurality of pieces of operational information associated with the operation of the hydraulic excavator 101A that changes over time, calculates the operational state of the hydraulic excavator 101A based on the acquired operational information of the hydraulic excavator 101A, and controls the display device 165 based on the calculation results.

[0027] The operating state calculation device 1 includes a collection device 1a that collects multiple pieces of operation information associated with the operation of the hydraulic excavator 101A that changes over time, and a processing device 1b that calculates the operating state of the hydraulic excavator 101A based on the multiple pieces of collected operation information. In this embodiment, a machine control device 110 provided in the hydraulic excavator 101A functions as the collection device 1a, and a management server 150 provided in the management facility functions as the processing device 1b. In this embodiment, the operation information collected by the collection device 1a is transmitted to the processing device 1b via a communication line 109, and the processing device 1b executes processing to calculate the operating state of the hydraulic excavator 101A. The calculation of the operating state (detection of the operating state) is performed by calculating a feature vector, which will be described later.

[0028] The processing device 1b calculates a feature vector using a plurality of pieces of operation information as feature quantities, and executes classification processing to assign a classification ID to the feature vector based on the classification model 1h. The processing device 1b calculates damage suffered by the hydraulic excavator 101A based on the plurality of pieces of operation information, and stores in the HDD 156 a damage operation database 1u that links the time, the classification ID assigned to the calculated feature vector, and the calculated damage.

[0029] The processing device 1b calculates a damage statistic for each classification ID as a representative value based on the time, classification ID, and damage contained in the damage behavior database 1u. The processing device 1b generates a sentence (hereinafter also referred to as a "note sentence") that expresses the operating state of the hydraulic excavator 101A corresponding to the classification ID by connecting words corresponding to the magnitudes of multiple feature quantities that make up the representative feature vector linked to the classification ID. The processing device 1b outputs to the display device 165 behavior information that associates the note sentence with the representative value of damage calculated for the classification ID linked to the representative feature vector used to generate the note sentence. The display device 165 outputs the behavior information input from the processing device 1b as an image.

[0030] As a result, a manager who views the image displayed on the display device 165 can easily know the operating state that will cause damage to the hydraulic excavator 101A from the relationship between the annotation that indicates the operating state of the hydraulic excavator 101A and the representative value of the damage. Each function of the operating state calculation device 1 will be described in detail below.

[0031] -First Collection Section and Second Collection Section- The collection device 1a has functions as a first collection unit 1f and a second collection unit 1g. The first collection unit 1f collects first operation information used to classify the operating state (feature vector) of the hydraulic excavator 101A. The second collection unit 1g collects second operation information used to calculate damage suffered by the hydraulic excavator 101A. The first operation information and the second operation information include sensor information detected by the detection devices 121 to 124 and control information generated by the machine control device 110 to control each part of the hydraulic excavator 101A. The first operation information and the second operation information may be the same type of information or different types of information.

[0032] The first collection unit 1f and the second collection unit 1g collect multiple pieces of sensor information for each time by linking the time-series changing sensor information with the time when the sensor information was detected and storing the information in the HDD 116. The first collection unit 1f and the second collection unit 1g collect multiple pieces of control information for each time by linking the time-series changing control information with the time when the control information was generated and storing the information in the HDD 116.

[0033] In this embodiment, an example is described in which the machine control device 110 functions as the collection device 1a, but the machine control device 110 and the collection device 1a may be provided separately. When the collection device 1a is provided separately from the machine control device 110, the collection device 1a may collect sensor information and control information by acquiring sensor information and control information flowing in a network such as a Controller Area Network (CAN) connected to various devices including the machine control device 110 and the collection device 1a.

[0034] The processing device 1b receives the first operation information and second operation information collected by the collection device 1a. The processing device 1b executes various processes described below based on the acquired first operation information and second operation information. The processing device 1b has functions as a classification processing unit 1i, a damage calculation processing unit 1j, a linking processing unit 1k, a statistical processing unit 1m, an annotation text generation unit 1n, and a display processing unit 1o. The HDD 156 of the processing device 1b functions as a storage unit that stores the classification model 1h and the damage action database 1u.

[0035] -Classification processing section- The classification processing unit 1i calculates a plurality of feature vectors using a plurality of types of time-divided operation information as feature quantities, and executes a classification process to classify the calculated feature vectors into one of a plurality of predetermined clusters. In the classification process, the classification processing unit 1i assigns a classification ID to the classified feature vector, which identifies the cluster to which the feature vector belongs. The processing contents of the classification processing unit 1i will be specifically described below.

[0036] The classification processing unit 1i calculates a feature vector using a plurality of pieces of operational information as feature quantities at each of a plurality of times. The classification processing unit 1i assigns a classification ID to the feature vector calculated at each of a plurality of times based on a classification model 1h stored in the HDD 156. The classification model 1h associates representative feature vectors of a plurality of clusters with a plurality of classification IDs. The classification processing unit 1i assigns, to the calculated feature vector, a classification ID associated with a representative feature vector similar to the calculated feature vector.

[0037] The representative feature vector is a multidimensional parameter representing the operating state of the hydraulic excavator 101A, and is a representative value of a subset (cluster) of similar feature vectors representing the operating state of the hydraulic excavator 101A. The representative feature vector is a vector made up of multiple feature quantities, and can also be expressed as a representative feature coordinate value. The classification ID(i) is an identification number that identifies the representative feature vector (i=1, 2, 3, ..., j).

[0038] The feature quantities may be the sensor information itself output from the detection devices 121 to 124, or information calculated based on the sensor information output from the detection devices 121 to 124. Furthermore, the feature quantities may be the control information itself generated by the machine control device 110, or information calculated based on the control information. Furthermore, the feature quantities may be information calculated based on both the sensor information and the control information. The information used for the feature quantities is operation information associated with the operation of the hydraulic excavator 101A, which changes over time. The calculation process of the operation information based on the sensor information and the control information may be performed by the classification processing unit 1i or the first collection unit 1f.

[0039] For example, the feature quantities can be operation information such as boom angle xi, arm angle yi, bucket angle zi, boom cylinder force mi, arm cylinder force ni, bucket cylinder force ki, etc. The "i" attached to the symbols (x, y, z, m, n, k) representing each feature quantity is a number indicating the classification ID.

[0040] The boom cylinder force mi is a cylinder output (cylinder thrust) calculated based on the pressure in the bottom-side oil chamber and the pressure in the rod-side oil chamber of the boom cylinder 11a. The arm cylinder force ni is a cylinder output (cylinder thrust) calculated based on the pressure in the bottom-side oil chamber and the pressure in the rod-side oil chamber of the arm cylinder 12a. The bucket cylinder force ki is a cylinder output (cylinder thrust) calculated based on the pressure in the bottom-side oil chamber and the pressure in the rod-side oil chamber of the bucket cylinder 13a.

[0041] The number of dimensions of the representative feature vector can be set arbitrarily. If the number of features constituting the representative feature vector is n, the representative feature vector will have n dimensions (n=6 in the illustrated example).

[0042] The classification model 1h is generated in advance by a well-known clustering method such as k-means based on operational test data of the hydraulic excavator 101A. k-means is a method of grouping similar feature vectors into a number of clusters according to a preset division number, and assigning a classification ID to identify each cluster. As a result, multiple feature vectors are classified according to the set division number, and the classified feature vectors are assigned the classification ID of the cluster to which they belong. The representative feature vector is a vector that represents the centroid coordinate value (cluster center) of multiple feature vectors (feature coordinate values) classified into the same cluster, and indicates a representative operational state of the hydraulic excavator 101A.

[0043] An example of a method for generating a classification model will be described with reference to Fig. 3A. Note that a six-dimensional feature vector consisting of six feature amounts needs to be expressed in a six-dimensional space, but six-dimensional space cannot be expressed in a drawing. For this reason, in Fig. 3A, two of the six feature amounts are selected, and the position coordinates of the feature vector 4a and representative feature vectors 4b, 4c, and 4d are shown in a two-dimensional orthogonal coordinate system. Also in Fig. 3A, the feature vector 4a at each time is indicated by a circle, and the representative feature vectors (representative feature coordinate values) 4b, 4c, and 4d are indicated by an x.

[0044] The clustering method is a method for dividing (classifying) adjacent point groups (collections of points representing feature vectors) 4f, 4g, and 4h into a predetermined number of divisions based on these feature vectors (feature coordinate values) 4a. This method outputs representative feature coordinate values (representative feature vectors) 4b, 4c, and 4d of the divided point groups 4f, 4g, and 4h, along with a classification ID for identifying them.

[0045] In the illustrated example, a representative feature vector (x1, y1, z1, m1, n1, k1) is determined based on point group 4f and assigned a classification ID of 1. Similarly, a representative feature vector (x2, y2, z2, m2, n2, k2) is determined based on point group 4g and assigned a classification ID of 2. Similarly, a representative feature vector (x3, y3, z3, m3, n3, k3) is determined based on point group 4h and assigned a classification ID of 3. Note that the representative feature vector is not limited to being a centroid coordinate value. The representative feature vector can be expressed using various representative feature coordinate values, such as an average value.

[0046] The classification model (also referred to as the time-division state separation model) 1h generated in the above manner will store representative feature vectors (representative feature coordinate values) representing various operating states of the hydraulic excavator 101A in the learning phase (for example, characteristic postures and sensor information, control information, etc. in various operations such as excavation operations and soil dumping operations) together with classification IDs.

[0047] A specific example of classification processing by the classification processing unit 1i will be described with reference to Fig. 3B. Note that a six-dimensional feature vector consisting of six feature amounts needs to be expressed in six-dimensional space, but six-dimensional space cannot be expressed in a drawing. For this reason, in Fig. 3B, as in Fig. 3A, two of the six feature amounts are selected, and the position coordinates of feature vector 4e and representative feature vectors 4b, 4c, and 4d are schematically shown in a two-dimensional orthogonal coordinate system.

[0048] The classification processing unit 1i calculates a feature vector (feature coordinate values) 4e based on the multiple pieces of first operation information acquired from the first collection unit 1f. The classification processing unit 1i searches for a representative feature vector (representative feature coordinate values) 4d that is closest to the calculated feature vector (feature coordinate values) 4e from within the classification model 1h. The classification processing unit 1i assigns the classification ID (3) of the searched representative feature vector 4d to the calculated feature vector 4e. In other words, the classification processing unit 1i classifies the feature vector 4e into the classification ID (3). The classification processing unit 1i determines classification IDs for the representative feature vectors calculated for each of the multiple time periods, generates a classification ID time-series table 5a (see FIG. 5), and stores it in the HDD 156.

[0049] -Damage Calculation Processing Unit- The damage calculation processor 1j shown in Fig. 2 executes calculation processing of damage suffered by the hydraulic excavator 101A. The damage calculation processor 1j calculates the damage suffered by the hydraulic excavator 101A at multiple time points based on multiple pieces of second operation information acquired from the second collector 1g. The damage calculation processor 1j also detects sections (periods) on the time axis in which the damage suffered by the hydraulic excavator 101A is significant. The damage that the damage calculation processor 1j targets is, for example, damage caused by stress fatigue damage suffered by predetermined parts of the structure, such as the boom 11 and arm 12, and hydraulic actuators, such as hydraulic cylinders and hydraulic motors.

[0050] An example of damage calculation processing is the method described in Patent Document 1. This method is a method for detecting the start and end times of repeated stress using stress values measured by a strain gauge (detection device) attached to the dump truck 101B.

[0051] An example of damage calculation processing by the damage calculation processing unit 1j will be described with reference to Figure 4. Figure 4 shows time-series changes in stress, stress amplitude, and fatigue damage rate. Here, an example will be described in which the work machine 101 is a dump truck 101B. A detection device 124 that detects strain is attached to the dump truck 101B. The vertical axis of the stress graph (3a) in Figure 4 represents stress, the vertical axis of the stress amplitude graph (3b) represents stress amplitude, and the vertical axis of the fatigue damage rate graph (3c) represents fatigue damage rate. The horizontal axis of the stress graph (3a), stress amplitude graph (3b), and fatigue damage rate graph (3c) is common and represents time.

[0052] The stress waveform 3h shown in Figure 4 is a graph of the detection results from the detection device 124. Fatigue damage depends on the amplitude of the cyclic stress, not the absolute value of the stress. The stress waveform 3h shows the absolute value of the stress, and it is therefore impossible to determine where significant fatigue damage has occurred. Therefore, the damage calculation processing unit 1j detects the amplitude and wavelength of the cyclic stress from the stress waveform 3h and detects the section where significant fatigue damage has occurred. This method detects the wavelength of the stress by improving the rainflow method, a type of stress cycle counting method. The stress amplitudes 3i and 3j are the amplitude and wavelength of the cyclic stress detected from the stress waveform 3h using this method. The stress amplitudes 3i and 3j are rectangular, with the vertical width indicating the amplitude of the detected cyclic stress and the horizontal width indicating the wavelength of the detected cyclic stress. The time corresponding to the vertex on the left side of the base of the rectangular stress amplitudes 3i and 3j indicates the start time of the cyclic stress. The time corresponding to the vertex on the right side of the base of the rectangular stress amplitudes 3i and 3j indicates the end time of the cyclic stress.

[0053] The work of transporting soil and sand to the dump truck 101B is carried out in the following order: an empty load transfer process (3d) in which the dump truck 101B moves in an empty state; a loading process (3e) in which soil and sand is loaded into the stopped dump truck 101B in multiple batches by the hydraulic excavator 101A; a loading and transfer process (3f) in which the dump truck 101B moves in a loaded state; and an earth discharge process (3g) in which the loaded soil and sand is discharged.

[0054] The stress amplitude 3i indicates the amplitude and wavelength of the repeated stress that the dump truck 101B receives in the loading process (3e). The stress amplitude 3j indicates the amplitude and wavelength of the repeated stress that the dump truck 101B receives in the soil unloading process (3g). Using this method, the damage calculation processing unit 1j can detect the magnitude of damage received by the hydraulic excavator 101A (the magnitude of the amplitude of the repeated stress) and the section in which the hydraulic excavator 101A received damage (the period from the start time to the end time of the amplitude of the repeated stress).

[0055] The fatigue damage rate is the fatigue damage sustained by the hydraulic excavator 101A per unit time. The damage calculation processor 1j calculates the fatigue damage rate based on the amplitude and wavelength of the repeated stress shown in the stress amplitude graph (3b). In other words, the damage calculation processor 1j calculates the fatigue damage rate as the damage sustained by the dump truck 101B. The damage calculation processor 1j generates a damage (fatigue damage rate) time series table 5b (see FIG. 5) calculated for each of a plurality of times, and stores the table in the HDD 156. Note that the damage calculation processing method has been described using the dump truck 101B as an example, but the same method can also be used for the hydraulic excavator 101A and the work robot 101C. Furthermore, the damage calculation processing method is not limited to this method.

[0056] -Linking processing section- 2 executes a linking process that links the processing results of the damage calculation processing unit 1j with the processing results of the classification processing unit 1i. The linking processing unit 1k generates a damage operation database 1u that links the classification ID assigned to the feature vector by the classification processing unit 1i, the damage (fatigue damage rate) calculated by the damage calculation processing unit 1j, and the time when the operation information used in the calculation of the feature vector and damage was detected, and stores the database in the HDD 156.

[0057] An example of the linking process by the linking processing unit 1k will be described with reference to Fig. 5. As described above, Fig. 5 shows the classification ID time series table 5a generated by the classification process of the classification processing unit 1i and the damage time series table 5b generated by the damage calculation process of the damage calculation processing unit 1j. The horizontal axis of the classification ID time series table 5a indicates time, and the vertical axis indicates the classification ID. In the classification ID time series table 5a, the classification IDs classified in chronological order are indicated by black dots. In the example shown in Fig. 5, the number of cluster divisions of the classification model 1h is 2500, and there are 2500 classification IDs.

[0058] The horizontal axis of the damage time series table 5b represents time, and is synchronized with the time axis of the classification ID time series table 5a. The vertical axis of the damage time series table 5b represents the fatigue damage rate (damage) caused by the boom cylinder force of the hydraulic excavator 101A. The fatigue damage rate shown in Fig. 5 is calculated based on the stress acting on parts related to the boom cylinder 11a, such as the cylinder pin, due to the boom cylinder force.

[0059] The linking processor 1k generates a damage operation database 1u linking time, classification ID, and fatigue damage rate based on the classification ID time series table 5a and the damage time series table 5b, and stores the database in the HDD 156. For example, the linking processor 1k associates and stores the classification ID "652" (see point 5c) and fatigue damage rate "0.0348" (see point 5d) at the time "513.5" seconds indicated by the dashed line. The linking processor 1k associates and stores the classification ID and fatigue damage rate in chronological order for all measurement times stored in the HDD 156. The measurement times are the times when the collecting device 1a acquires operation information at a control period of about 100 msec to 1 sec.

[0060] Although the fatigue damage rate calculated based on the boom cylinder force has been described as an example of damage, a fatigue damage rate calculated based on other hydraulic cylinder forces (arm cylinder force, bucket cylinder force) may also be linked to a classification ID. Also, a fatigue damage rate calculated based on the detection results of the detection device (strain gauge) 124 may be linked to a classification ID, or a physical quantity other than the fatigue damage rate that serves as an index of damage may be linked to a classification ID.

[0061] -Statistical processing section- 2 executes statistical processing of damage for each classification ID based on the time, classification ID, and damage contained in the damage action database 1u. In the statistical processing, the statistical processing unit 1m calculates statistical quantities such as the average value, maximum value, most frequent value, median value, and cumulative value of damage for each classification ID as representative values.

[0062] An example of statistical processing of damage by the statistical processing unit 1m will be described with reference to Figure 6. Figure 6 is a diagram showing an example of time-series changes in classification ID, fatigue damage rate, and cumulative value of fatigue damage rate in a specified section (hereinafter also referred to as cumulative damage). Figure 6(a) shows the time-series changes in classification ID, Figure 6(b) shows the time-series changes in fatigue damage rate, and Figure 6(c) shows the cumulative value of fatigue damage rate in the section with classification ID (5). As shown in Figure 6, classification ID and damage (here, fatigue damage rate) change from moment to moment.

[0063] As shown in FIG. 6(a), in the illustrated example, the classification ID changes from (3) to (5) and then to (2) over time. As shown in FIG. 6(b), the fatigue damage rate in the section with classification ID (5) is higher than the sections with classification ID (3) and the section with classification ID (2). Also, as shown in FIG. 6(b), the fatigue damage rate changes over time even in sections with the same classification ID. One of the objectives of this embodiment is to calculate the operating state (feature vector) of the hydraulic excavator 101A when major damage occurs. The operating state of the hydraulic excavator 101A corresponds to the classification ID.

[0064] The statistical processing unit 1m extracts sections with the same classification ID and calculates statistical quantities such as the average, maximum, most frequent, median, and cumulative value of damage occurring within the section with the same classification ID as representative values. For example, when calculating the maximum value of fatigue damage rate within a section with the same classification ID as a statistical quantity, the maximum value 6c of damage occurring at a sharp peak can be detected with high sensitivity, as shown in Figure 6(b). Furthermore, the statistical processing unit 1m may calculate the cumulative value of fatigue damage rate (cumulative damage) within a section with the same classification ID as a statistical quantity, as shown in Figure 6(c).

[0065] The same classification ID may be set in multiple intervals. For example, classification ID (5) may be set in an interval (time period) other than that shown in FIG. 6. When the same classification ID is set in multiple intervals (time periods), the statistical processing unit 1m may calculate the damage statistics for each interval, or may calculate the damage statistics by treating each interval as a single interval. Hereinafter, the damage statistics (representative value) will also be referred to as the damage amount.

[0066] The statistical processing unit 1m determines whether the amount of damage is equal to or greater than the representative threshold. If the statistical processing unit 1m determines that the amount of damage is less than the representative threshold, it excludes the classification ID of the section (time period) in which the damage amount was calculated from the annotation generation process. If the statistical processing unit 1m determines that the amount of damage is equal to or greater than the representative threshold, it extracts the classification ID of the section (time period) in which the damage amount was calculated as the annotation generation process. The representative threshold is a threshold for determining whether the amount of damage affects the lifetime (the amount of use until the equipment or parts are replaced) of the devices and parts that make up the hydraulic excavator 101A, and is stored in advance in the ROM 152. The representative threshold may be set using a life curve (SN diagram) or the like, or may be set manually by visually observing the condition of the hydraulic excavator 101A through experiments, etc.

[0067] As described above, in this embodiment, there are 2,500 classification IDs (see FIG. 5). However, most of the classification IDs correspond to operations that do not affect the lifetime of the devices and components that make up the hydraulic excavator 101A. For this reason, it is not desirable to subject such classification IDs to the annotation generation process described below in terms of the load on the calculation process. Furthermore, if a large number of annotations are generated, similar annotations may be generated, making it difficult to determine which operating states will cause significant damage. Furthermore, since it is necessary to generate a unique annotation for each of the 2,500 classification IDs, there is a risk that the annotations may become long.

[0068] Therefore, the statistical processing unit 1m extracts the classification IDs for which the damage amount is equal to or greater than the representative threshold, and excludes the classification IDs for which the damage amount is less than the representative threshold from the annotation generation process. This reduces the processing load on the processing unit 1b and prevents the annotation from becoming too long.

[0069] FIG. 7 is a graph showing the relationship between classification ID and damage amount. The vertical axis shows the maximum damage in the section of the same classification ID as an example of damage amount. The horizontal axis shows classification IDs arranged in descending order of damage amount. In the example shown, the top 10 classification IDs with the greatest damage amount are extracted and shown. In the example shown, classification ID (20) has the greatest damage amount.

[0070] Here, if a graph such as that shown in FIG. 7 is displayed on the display device 165, the manager can know the classification IDs that have a large amount of damage. However, simply displaying a graph such as that shown in FIG. 7 on the display device 165 does not allow the manager to know the operating state of the hydraulic excavator 101A in which large damage will occur. Furthermore, if the classification ID time series table 5a and the damage time series table 5b shown in FIG. 5 are displayed on the display device 165, the manager can check the time series changes in the classification ID and damage, and the classification IDs of sections with large damage. However, simply displaying the classification ID time series table 5a and the damage time series table 5b on the display device 165 does not allow the manager to know the operating state of the hydraulic excavator 101A in which large damage will occur.

[0071] Therefore, the processing device 1b according to this embodiment generates a comment that explains the operating state of the hydraulic excavator 101A for each classification ID.

[0072] -Annotation generation section- The annotation generation unit 1n shown in Figure 2 classifies the extracted multiple classification IDs into multiple groups, identifies features to be used for generating annotations based on the differences between representative feature vectors linked to multiple classification IDs classified into the same group, and generates annotations that represent the operating status of the hydraulic excavator 101A corresponding to the classification ID by connecting words that correspond to the magnitudes of the identified features.

[0073] An example of the flow of processing executed by the annotation sentence generation unit 1n will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the flow of processing executed by the annotation sentence generation unit 1n. As shown in Fig. 8, in step S100, the annotation sentence generation unit 1n acquires the processing result of the statistical processing unit 1m and the classification model 1h, and proceeds to step S105.

[0074] In step S105, the annotation generator 1n executes a hierarchical clustering process, in which the annotation generator 1n classifies the classification IDs into a plurality of hierarchically structured groups based on the representative feature vectors.

[0075] The hierarchical clustering process will be described in detail with reference to Fig. 9. The hierarchical clustering process is known as a type of hierarchical classification process. As shown in Fig. 9, the annotation generation unit 1n classifies the classification IDs extracted by the statistical processing unit 1m in a hierarchical structure.

[0076] In Fig. 9(a), classification ID(5), classification ID(6), classification ID(7), classification ID(20), classification ID(28), and classification ID(38), represented by circles and numbers, indicate the position coordinates (representative feature coordinate values) of the representative feature vectors of the classification IDs extracted by the statistical processing unit 1m. Note that in Fig. 9(a), as in Fig. 3A and Fig. 3B, two of the multiple feature amounts are selected, and the position coordinates of the representative feature vectors are schematically shown in a two-dimensional orthogonal coordinate system.

[0077] 9(a), the annotation generator 1n reads out the representative feature vectors for the classification IDs with large amounts of damage extracted by the statistical processor 1m from the classification model 1h. The annotation generator 1n classifies the classification IDs into multiple groups based on the distance L between the classification IDs (the distance between the representative feature coordinate values).

[0078] The annotation generator 1n groups the classification IDs with the closest distance one by one in order. In the example shown in FIG. 9(a), the annotation generator 1n classifies classification ID (20) and classification ID (28), which is closest to classification ID (20), into the same subgroup (A). The annotation generator 1n also stores the distance L between classification ID (20) and classification ID (28) as an attribute of the subgroup (A). Similarly, the annotation generator 1n classifies classification ID (5) and classification ID (38), which is closest to classification ID (5), into the same subgroup (B), and classifies classification ID (6) and classification ID (7), which is closest to classification ID (6), into the same subgroup (C).

[0079] The annotation generator 1n classifies the multiple lower groups into multiple medium-level groups based on the distance L between the multiple lower groups. The medium-level groups are groups hierarchically higher than the lower groups. The representative feature coordinate value of a lower group is, for example, the barycentric coordinate value of the representative feature coordinate values of the two classification IDs that make up the lower group, and can also be expressed as a representative feature vector of the lower group.

[0080] In addition, if there is a classification ID that is not classified into a lower group, the annotation text generation unit 1n determines whether to classify the classification ID that is not classified into a lower group and the lower group into the same intermediate group based on the distance L between the classification ID and the lower group.

[0081] 9(a), the annotation generator 1n classifies a lower group (A) and a lower group (B) that is closest to the lower group (A) into the same middle group (D). The annotation generator 1n also stores the distance L between the lower group (A) and the lower group (B) as an attribute of the middle group (D).

[0082] The annotation generator 1n classifies the multiple middle-level groups into multiple upper-level groups based on the distance L between the multiple middle-level groups. An upper-level group is a group that is hierarchically higher than the middle-level group. The representative feature coordinate value of a middle-level group is, for example, the barycentric coordinate value of the representative feature coordinate values of the two lower-level groups that make up that middle-level group, and can also be expressed as a representative feature vector of the middle-level group.

[0083] In addition, if there are classification IDs that are not classified into lower groups and middle groups, or lower groups that are not classified into middle groups, the annotation text generation unit 1n determines whether to classify them into the same upper group based on the distance L between such classification IDs or lower groups and the middle group.

[0084] 9(a), the annotation generator 1n classifies a middle group (D) and a lower group (C) that is closest to the middle group (D) into the same upper group (E). The annotation generator 1n also stores the distance L between the middle group (D) and the lower group (C) as an attribute of the upper group (E).

[0085] 9(b) is a tree diagram showing the results of the hierarchical clustering process performed by the annotation text generation unit 1n. The vertical direction of the tree diagram represents distance L (for example, the distance L between two classification IDs belonging to the same lower group), and the ends are "leaves." In this embodiment, the classification IDs extracted by the statistical processing unit 1m are "leaves." In the example shown, classification ID (28) and classification ID (20) are combined to generate lower group (A), classification ID (38) and classification ID (5) are combined to generate lower group (B), classification ID (6) and classification ID (7) are combined to generate lower group (C), lower group (A) and lower group (B) are combined to generate middle group (D), and middle group (D) and lower group (C) are combined to generate upper group (E).

[0086] In this way, the annotation generator 1n classifies the classification IDs into a plurality of hierarchically structured groups based on the representative feature vectors in the hierarchical clustering process (step S105). As shown in Fig. 8, when the hierarchical clustering process (step S105) is completed, the process proceeds to step S110.

[0087] In step S110, the annotation generator 1n executes a process for identifying branching factors based on the results of the hierarchical clustering process. The feature that caused branching between the same groups can be identified by the distance L stored as an attribute of the group (for example, the distance L between two classification IDs belonging to a certain lower group). The distance L stored as an attribute of the group is calculated based on the classification IDs belonging to the group or the difference between the representative feature vectors of the group.

[0088] The annotation generator 1n identifies branching factors based on the difference between two representative feature vectors of classification IDs or groups within the same group, i.e., the difference in feature values (elements). Specifically, the annotation generator 1n identifies a feature value Ca as a branching factor at the first level based on the difference between the representative feature vectors of two middle-level groups (or a middle-level group and a lower-level group, or a middle-level group and a classification ID) classified into the same upper level group. The annotation generator 1n also identifies a feature value Cb as a branching factor at the second level based on the difference between the representative feature vectors of two lower-level groups (or a lower-level group and a classification ID) classified into the same middle level group. Furthermore, the annotation generator 1n identifies a feature value Cc as a branching factor at the third level based on the difference between the representative feature vectors of two classification IDs belonging to the same lower level group.

[0089] In step S110, when the identification of the features that caused the branching is completed, the process proceeds to step S115. In step S115, the annotation sentence generation unit 1n selects words corresponding to the magnitudes of the features Ca, Cb, and Cc that caused the branching identified in step S110.

[0090] An example of the process of selecting words corresponding to the magnitude of the identified feature amount (the process of selecting words that are branching factors) will be described with reference to Fig. 10. In the example shown in Fig. 10, among the feature amounts constituting the representative feature vector, the arm angle differs between classification ID (28) and classification ID (20), and therefore the arm angle is identified as feature amount Cc. Note that the difference between the feature amounts other than the arm angle constituting the representative feature vectors of classification ID (28) and classification ID (20) is smaller than the difference between the arm angles.

[0091] Words corresponding to the arm angle, which is a feature, are stored in HDD 156. For example, the word "arm angle vertical" that is selected when the arm angle is within a first angle range, the word "arm angle Max" that is selected when the arm angle is within a second angle range, and the word "arm angle Min" that is selected when the arm angle is within a third angle range are stored in HDD 156. Note that the first angle range, the second angle range, and the third angle range are non-overlapping angle ranges.

[0092] In the illustrated example, in step S110, it is determined that the branching factor (feature Cc) between category ID (28) and category ID (20), which belong to subgroup (A), is the arm angle. Therefore, in step S115, the annotation generator 1n selects from HDD 156 a phrase corresponding to the magnitude of the arm angle, which is the feature Cc constituting the representative feature vector of category ID (28), and a phrase corresponding to the magnitude of the arm angle, which is the feature Cc constituting the representative feature vector of category ID (20). Because the arm angle constituting the representative feature vector of category ID (28) is within the first angle range, the annotation generator 1n selects the phrase "arm angle vertical." Because the arm angle constituting the representative feature vector of category ID (20) is within the second angle range, the annotation generator 1n selects the phrase "arm angle Max."

[0093] In the illustrated example, in step S110, the branching factor (feature value Cb) between the lower group (A) and the lower group (B) belonging to the middle group (D) is identified as the sign of the turning force. The turning force is a feature value (operation information) calculated based on the pressure of the turning motor.

[0094] Words corresponding to the sign of the turning force, which is a feature quantity, are stored in HDD 156. For example, the word "left" selected when the turning force is a positive value, and the word "right" selected when the turning force is a negative value, etc. are stored in HDD 156. Note that with regard to the turning force, a force that turns the turning unit 3 leftward is represented by a positive value, and a force that turns the turning unit rightward is represented by a negative value.

[0095] In step S115, the annotation text generation unit 1n selects from HDD 156 a phrase corresponding to the magnitude of the turning force, which is the feature Cb constituting the representative feature vector of the subgroup (A), and a phrase corresponding to the magnitude of the turning force, which is the feature Cb constituting the representative feature vector of the subgroup (B).

[0096] Since the rotation force constituting the representative feature vector of the lower group (A) is a positive value (a value greater than 0), the annotation sentence generator 1n selects the word "left." Since the rotation force constituting the representative feature vector of the lower group (B) is a negative value (a value smaller than 0), the annotation sentence generator 1n selects the word "right."

[0097] In the illustrated example, in step S110, the branching factor (feature amount Ca) between the middle group (D) and the lower group (C) (see FIG. 9) belonging to the upper group (E) is identified as the absolute value of the turning force.

[0098] Words corresponding to the absolute value of the turning force, which is a feature, are stored in HDD 156. For example, the word "side contact" that is selected when the absolute value of the turning force is equal to or greater than the side contact threshold, and the word "turning" that is selected when the turning force is less than the side contact threshold are stored in HDD 156.

[0099] Here, "side hitting" refers to the operation of operating the rotating body 3 of the hydraulic excavator 101A to bring the side of the working device 4 into contact with a stone, wall, etc. and push it, and "rotation" refers to the operation of rotating the rotating body 3 of the hydraulic excavator 101A relative to the running body 2.

[0100] In step S115, the annotation generator 1n selects from the HDD 156 a phrase corresponding to the magnitude of the absolute value of the turning force, which is the feature amount Ca constituting the representative feature vector of the middle group (D), and a phrase corresponding to the magnitude of the absolute value of the turning force, which is the feature amount Ca constituting the representative feature vector of the lower group (C). Because the absolute value of the turning force constituting the representative feature vector of the middle group (D) is equal to or greater than the lateral contact threshold, the annotation generator 1n selects the phrase "lateral contact." Because the absolute value of the turning force constituting the representative feature vector of the lower group (C) is less than the lateral contact threshold, the annotation generator 1n selects the phrase "turn."

[0101] 8, when the process of selecting the words and phrases of the branching causes is completed in step S115, the process proceeds to step S120. In step S120, the annotation sentence generation unit 1n generates an annotation sentence representing the operating state of the hydraulic excavator 101A by connecting the words and phrases selected in step S115 for each classification ID (the part corresponding to the leaves of the tree diagram).

[0102] The annotation sentence is generated by combining words (linguistic expressions) for each branching factor. In the example shown in FIG. 10, the annotation sentence for the classification ID (28) belonging to the upper group (E), the middle group (D), and the lower group (A) is "left side-attach, arm angle vertical," which is a combination of the words "side-attach" selected based on the branching factor in the upper group (E), the words "left" selected based on the branching factor in the middle group (D), and the words "arm angle vertical" selected based on the branching factor in the lower group (A). The order in which the words are connected can be arbitrarily set using the input device 161. Therefore, for example, the words can be connected so that the annotation sentence becomes "arm angle vertical, left side-attach."

[0103] As shown in Fig. 8, when the annotation text generation process is completed in step S120, the process shown in the flowchart of Fig. 8 is completed, and the display processing by the display processing unit 1o shown in Fig. 2 is executed. The display processing unit 1o outputs, to the display device 165, action information (display image) that associates the annotation text generated by the annotation text generation process with the damage amount calculated for the classification ID linked to the representative feature vector used to generate the annotation text. The display device 165 displays the action information (display image) input from the processing device 1b on the display screen 165a.

[0104] Various display modes can be adopted for the display image associating the annotation with the amount of damage. FIG. 11 shows a bar graph as an example of a display image associating the annotation with the amount of damage. As shown in FIG. 11, the display processing unit 1o may display a bar graph on the display screen 165a of the display device 165, in which the classification IDs are arranged from left to right in descending order of the amount of damage. The horizontal axis displays the classification IDs and the annotations corresponding to the classification IDs. The vertical axis represents the amount of damage corresponding to the classification ID. The bar graph in FIG. 11 is the bar graph in FIG. 7 to which annotations have been added.

[0105] As described above, the manager cannot know in what operating state the hydraulic excavator 101A will suffer major damage if the bar graph shown in Fig. 7 is simply displayed on the display device 165. In contrast, in this embodiment, a bar graph with an annotation explaining the operating state is displayed on the display device 165 as shown in Fig. 11, so that the manager can easily know in what operating state the hydraulic excavator 101A will suffer major damage.

[0106] The display processing unit 1o may hierarchically display the words and phrases constituting the annotation on the display device 165. For example, the display processing unit 1o may display a tree diagram as shown in FIG. 10 on the display device 165. A display representation using a tree diagram makes it possible to grasp similar actions. Note that the tree diagram shown in FIG. 10 does not represent the amount of damage. Therefore, it is preferable that the display processing unit 1o change the display mode of the branches connected to the leaves in the tree diagram depending on the magnitude of the representative value of the damage. For example, as shown in FIG. 12, the display processing unit 1o may display the branches connected to the classification ID (leaf) thicker as the amount of damage increases. In this case, the display processing unit 1o displays an image 166a indicating the relationship between the thickness of the line and the magnitude of the amount of damage together with the tree diagram. This allows the administrator to easily understand the relationship between the operating state of the hydraulic excavator 101A represented by the annotation and the amount of damage.

[0107] 13, the line type of the branch connected to the classification ID (leaf) may be changed depending on the amount of damage. In this case, the display processing unit 1o displays an image 166b indicating the relationship between the line type and the magnitude of the amount of damage together with the tree diagram. This allows the administrator to easily understand the relationship between the operating state of the hydraulic excavator 101A expressed by the annotation and the amount of damage. Note that instead of changing the line type of the branch depending on the amount of damage, the color of the branch may be changed depending on the amount of damage.

[0108] According to the above-described embodiment, the following advantageous effects are achieved.

[0109] (1) The working machine motion state calculation device 1 includes a processing device 1b that calculates the motion state of a hydraulic excavator (work machine) 101A based on multiple pieces of operation information associated with the operation of the hydraulic excavator (work machine) 101A that changes over time, and a display device 165 as an output device that outputs motion information (images) that represent the motion state of the hydraulic excavator 101A calculated by the processing device 1b. The processing device 1b calculates, for multiple times, feature vectors using the multiple pieces of operation information as feature quantities. The processing device 1b assigns classification IDs to the feature vectors calculated for multiple times based on a classification model 1h in which a representative feature vector, which is a representative value of a cluster of feature vectors that represent the motion state of the hydraulic excavator 101A, is linked to a classification ID. The processing device 1b calculates damage suffered by the hydraulic excavator 101A for multiple times based on the multiple pieces of operation information. The processing device 1b stores a damage motion database 1u that links time, classification IDs assigned to the calculated feature vectors, and the calculated damage. The processing device 1b calculates a representative value (statistic) of damage for each classification ID based on the time, classification ID, and damage contained in the damage behavior database 1u. The processing device 1b generates a commentary (sentence) that represents the operating state of the hydraulic excavator 101A that corresponds to the classification ID by connecting words and phrases that correspond to the magnitudes of multiple feature quantities that make up the representative feature vector linked to the classification ID. The processing device 1b outputs to the display device 165 operation information (image) that associates the commentary that represents the operating state of the hydraulic excavator 101A with the representative value of damage calculated for the classification ID linked to the representative feature vector used to generate the commentary. The display device 165 outputs the operation information (image) input from the processing device 1b.

[0110] According to this configuration, a display image in which a representative value of damage and a comment indicating the operating state of the hydraulic excavator 101A are associated is displayed on the display device 165, so that a person (such as a manager) looking at the display device 165 can easily know the operating state that will damage the hydraulic excavator 101A. In other words, according to this embodiment, it is possible to provide a working machine operating state calculation device 1 that can easily know the operating state that will damage the hydraulic excavator 101A.

[0111] (2) The processing device 1b classifies the multiple classification IDs into multiple groups, and identifies features to be used for generating annotations based on the differences between the representative feature vectors associated with the multiple classification IDs classified into the same group. The processing device 1b generates annotations that represent the operating status of the hydraulic excavator 101A corresponding to the classification ID by connecting words corresponding to the magnitudes of the identified features.

[0112] The processing device 1b according to this embodiment classifies multiple classification IDs into multiple groups in a hierarchical structure based on their representative feature vectors. The processing device 1b calculates a representative feature vector for a lower group based on the representative feature vectors associated with multiple classification IDs belonging to the same lower group, and calculates a representative feature vector for a middle group based on the representative feature vectors for multiple lower groups belonging to the same middle group. The processing device 1b identifies a feature amount Ca based on the difference between the representative feature vectors for multiple middle groups belonging to the same upper group. The processing device 1b identifies a feature amount Cb based on the difference between the representative feature vectors for multiple lower groups belonging to the same middle group. The processing device 1b identifies a feature amount Cc based on the difference between the representative feature vectors associated with multiple classification IDs belonging to the same lower group. The processing device 1b generates an annotation describing the operating status of the hydraulic excavator 101A corresponding to the classification ID by combining a phrase corresponding to the magnitude of the identified feature amount Ca, a phrase corresponding to the magnitude of the identified feature amount Cb, and a phrase corresponding to the magnitude of the identified feature amount Cc.

[0113] In this way, the processing device 1b hierarchically classifies the classification IDs into multiple groups based on the representative feature vectors, and generates annotations by connecting words and phrases corresponding to the magnitudes of the features that make up the representative feature vectors used for the hierarchical classification. In other words, in this embodiment, annotations that reflect the results of the hierarchical classification process can be generated, and a hierarchical display image such as a tree diagram (see FIGS. 12 and 13) can be displayed on the display device 165.

[0114] (3) The processing device 1b causes the display device 165 to hierarchically display the words and phrases that make up the annotation sentence. The processing device 1b causes the display device 165 to display, for example, a tree diagram (see FIGS. 12 and 13) in which classification IDs are the leaves. By visualizing the differences in the action states that cause major damage in a hierarchical structure, a person (such as an administrator) viewing the display device 165 can easily know the action states that are similar to the action states that cause major damage.

[0115] (4) The processing device 1b changes the display mode of the branches connected to the leaves in the tree diagram depending on the magnitude of the representative value of the damage (see FIGS. 12 and 13). For example, the processing device 1b displays the branches thicker the larger the representative value of the damage (see FIG. 12). This allows a person (such as a manager) looking at the display device 165 to easily know the operating state that is damaging the hydraulic excavator 101A and the magnitude of that damage.

[0116] (5) The processing device 1b extracts classification IDs whose representative damage values are equal to or greater than a representative threshold, classifies the extracted classification IDs into multiple groups, and identifies features to be used in generating annotations based on the differences between representative feature vectors associated with the classification IDs in the same group. This prevents unnecessary processing for generating annotations. In other words, this embodiment reduces the computational load. It also prevents annotations from becoming too long.

[0117] <Modification of the first embodiment> The processing device 1b may calculate a representative value of damage for each work environment. The work environment may refer to, for example, the operator operating the hydraulic excavator 101A, the work area, the characteristics of the ground being excavated, and the like. The work environment can be identified by dividing the operation information based on work plan information input before work begins, location information using a Global Navigation Satellite System (GNSS), or information manually entered by the operator. The processing device 1b displays an image on the display device 165 that associates the amount of damage with a comment for each work environment, such as for each operator, work area, or the characteristics of the ground being excavated. This allows a person viewing the display device 165 (such as a manager) to understand the relationship between the amount of damage and the operating state according to the work environment. For example, by using the graph in FIG. 11, a work environment with a large amount of damage can be identified. If a common event affecting the location information and damage amount of the work machine 101 is identified, a more detailed analysis of the factors can be performed, such as by revising the travel route of the work machine 101.

[0118] Second Embodiment A processing device 1Bb according to a second embodiment of the present invention will be described with reference to Fig. 14. Fig. 14 is a functional block diagram of an operating state calculation device 1 according to the second embodiment. Note that in the second embodiment, components that are the same as or equivalent to those described in the first embodiment are given the same reference numerals, and differences will be mainly described.

[0119] In the first embodiment, the annotation statement generation unit 1n is configured to execute a series of processes (steps S105 to S120 in FIG. 8) for generating annotation statements for all classification IDs extracted by the statistical processing unit 1m each time the statistical processing unit 1m completes its statistical processing. In contrast, the annotation statement generation unit 1Bn in the second embodiment is configured not to execute a series of processes (steps S105 to S120 in FIG. 8) for generating annotation statements for classification IDs for which annotation statements have already been generated.

[0120] When the annotation generation unit 1Bn according to the second embodiment generates an annotation for a classification ID, the annotation is stored in association with the classification ID of the classification model 1Bh in the HDD (storage device) 156. That is, the classification model 1Bh according to the second embodiment stores the classification ID, the representative feature vector, and the generated annotation in association with each other. When the statistical processing by the statistical processing unit 1m is completed, the annotation generation unit 1Bn determines whether annotations have already been generated for all the classification IDs extracted by the statistical processing unit 1m.

[0121] Specifically, after executing the processing of step S100 in Figure 8, the annotation text generation unit 1Bn refers to the classification model 1Bh and determines, based on the classification ID extracted by the statistical processing unit 1m, whether the annotation text linked to the classification ID is stored in the classification model 1Bh of the HDD (storage device) 156.

[0122] If it is determined that the annotation linked to the classification ID is not stored in the classification model 1Bh of the HDD 156, the annotation generation unit 1Bn executes a series of processes (steps S105 to S120 in FIG. 8) to generate an annotation that indicates the operating state of the hydraulic excavator 101A that corresponds to the classification ID, as in the first embodiment. Furthermore, the annotation generation unit 1Bn associates the generated annotation with the classification ID and stores it in the classification model 1Bh of the HDD 156. In this case, the display processing unit 1o outputs to the display device 165 operation information that associates the generated annotation with the representative value of damage calculated for the classification ID associated with the generated annotation.

[0123] If it is determined that the annotation linked to the classification ID is stored in the classification model 1Bh of the HDD 156, the annotation generation unit 1Bn does not execute the series of processes (steps S105 to S120 in FIG. 8) for generating an annotation that indicates the operating state of the hydraulic excavator 101A corresponding to that classification ID. The annotation generation unit 1Bn selects an annotation stored in the classification model 1Bh of the HDD 156. The display processing unit 1o outputs to the display device 165 operation information that associates the selected annotation with the representative value of damage calculated for the classification ID linked to the selected annotation.

[0124] In this way, the processing device 1Bb according to the second embodiment stores the generated annotation in association with the classification ID. This eliminates the need to perform the annotation generation process again if an annotation has already been generated for the classification ID extracted by the statistical processing unit 1m. As a result, the processing load on the processing device 1Bb can be reduced.

[0125] Third Embodiment A processing device 1Cb according to a third embodiment of the present invention will be described with reference to Fig. 15. Fig. 15 is a functional block diagram of an operating state calculation device 1 according to the third embodiment. Note that in the third embodiment, components that are the same as or equivalent to those described in the second embodiment are given the same reference numerals, and differences will be mainly described.

[0126] In the classification processing unit 1i according to the first and second embodiments, as shown in FIG. 3B, an example has been described in which a calculated feature vector (feature coordinate values) 4e is assigned the classification ID of a representative feature vector (representative feature coordinate values) 4d that is closest to the calculated feature vector 4e. With this method, a classification ID is assigned even when the distance between the feature vector 4e and the representative feature vector 4d is large. As a result, there is a risk that the classification ID of the classification model 1h or 1Bh will not be appropriately assigned to the calculated feature vector 4e. This phenomenon occurs because the input data (features) were not sufficiently learned during the learning phase in which the classification models 1h or 1Bh were created.

[0127] Therefore, if a representative feature vector similar to the calculated feature vector is not stored in the classification model 1Ch, the processing device 1Cb according to the third embodiment shown in Fig. 15 updates the classification model 1Ch by adding a new representative feature vector and a new classification ID to the classification model 1Ch. This will be explained in detail below.

[0128] 15, the processing device 1Cb has a function as an update processing unit 1Cp that executes update processing of the classification model 1Ch. The classification processing unit 1Ci determines whether a representative feature vector similar to the calculated feature vector is stored in the classification model 1Ch. The classification processing unit 1Ci calculates the distance between the calculated feature vector and all representative feature vectors stored in the classification model 1Ch.

[0129] If all the calculated distances are equal to or greater than the dissimilarity threshold, the classification processing unit 1Ci determines that a representative feature vector similar to the calculated feature vector is not stored in the classification model 1Ch. If at least one of the calculated distances is less than the dissimilarity threshold, the classification processing unit 1Ci determines that a representative feature vector similar to the calculated feature vector is stored in the classification model 1Ch. An arbitrary value of the dissimilarity threshold is stored in advance in the ROM 152. The dissimilarity threshold may be determined based on statistics obtained from the distribution of classification IDs.

[0130] If it is determined that a representative feature vector similar to the calculated feature vector is not stored in the classification model 1Ch, the update processing unit 1Cp executes an update process for the classification model 1Ch. Specifically, the update processing unit 1Cp generates a new representative feature vector and a new classification ID based on the calculated feature vector. Furthermore, the update processing unit 1Cp associates the generated new representative feature vector with the new classification ID and stores them in the classification model 1Ch.

[0131] If it is determined that a representative feature vector similar to the calculated feature vector is stored in the classification model 1Ch, the update processing unit 1Cp does not execute the update processing of the classification model 1Ch.

[0132] The update processing unit 1Cp updates the classification model 1Ch by, for example, using the feature vector calculated by the classification processing unit 1Ci as a new representative feature vector. Note that the method of adding a new representative feature vector to the classification model 1Ch is not limited to this. For example, if the classification model 1Ch does not store representative feature vectors similar to each of multiple feature vectors calculated consecutively on the time axis by the classification processing unit 1Ci, the update processing unit 1Cp may update the classification model 1Ch by using the center of gravity of these feature vectors as a new representative feature vector.

[0133] In this way, if a representative feature vector similar to the calculated feature vector is not stored in the classification model 1Ch, the processing device 1Cb according to the third embodiment generates a new representative feature vector based on the calculated feature vector, associates it with a new classification ID, and stores it in the classification model 1Ch. This configuration allows for the generation of a new representative feature vector and a new classification ID associated therewith, even if a representative feature vector similar to the calculated feature vector is not stored in the classification model 1Ch. This makes it possible to appropriately execute various processes, such as classification processing, damage calculation processing, association processing, and annotation generation processing. As a result, annotations and damage amounts can be more appropriately presented to the administrator.

[0134] <Fourth embodiment> A processing device 1Cb according to a fourth embodiment of the present invention will be described with reference to Fig. 16 and Fig. 17. Fig. 16 is a functional block diagram of an operating state calculation device 1 according to the fourth embodiment, and Fig. 17 is a diagram showing an example of information output from a display device 125 and a speaker 126 provided in the operator's cab of a hydraulic excavator 101A. Note that in the fourth embodiment, components that are the same as or equivalent to those described in the third embodiment are given the same reference numerals, and differences will be mainly described.

[0135] In the first to third embodiments, an example was described in which the collection device 1a was provided on the work machine 101, and the processing devices 1b, 1Bb, and 1Cb were provided on the management facility. In contrast to this, in the fourth embodiment, the collection device 1Da and the processing device 1Db are provided on the work machine 101.

[0136] Furthermore, the processing device 1Db according to the fourth embodiment has, in addition to the functions of the processing device 1Cb described in the third embodiment, a function of sequentially determining whether or not major damage has occurred based on operation information sequentially collected by the first collection unit 1Df and the second collection unit 1Dg. Furthermore, the processing device 1Db has a function of generating a comment explaining the operating state of the work machine 101 at that time when major damage has occurred, and presenting the generated comment to the operator by voice. Note that in FIG. 16, the damage operation database 1u, statistical processing unit 1m, and comment generation unit 1Bn are omitted from the illustration, and only the characteristic functions of the processing device 1Db according to the fourth embodiment are illustrated. The operating state calculation device 1 according to the fourth embodiment will be described in detail below.

[0137] As shown in FIG. 16, the collecting device 1Da, functioning as the first collecting unit 1Df and the second collecting unit 1Dg, sequentially transmits the collected first operation information and second operation information to the processing device 1Db. The classification processing unit 1Di calculates a feature vector based on the input first operation information, and, with reference to the classification model 1Ch, assigns to the calculated feature vector a classification ID of a representative feature vector similar to the calculated feature vector. The damage calculation processing unit 1Dj calculates damage based on the input second operation information. The linking processing unit 1Dk links the classification ID classified by the classification processing unit 1Di with the damage calculated by the damage calculation processing unit 1Dj.

[0138] The processing device 1Db according to the fourth embodiment has functions as a damage determination unit 1Dq, an annotation text search unit 1Dr, and a sound output processing unit 1Ds. The damage calculation processing unit 1Dj calculates the damage suffered by the hydraulic excavator 101A. The damage determination unit 1Dq determines whether the damage calculated by the damage calculation processing unit 1Dj is equal to or greater than a damage threshold. In other words, the damage determination unit 1Dq determines whether damage equal to or greater than the damage threshold has occurred. The damage threshold is stored in advance in the ROM 152. The damage threshold may be changeable by an administrator operating the input device 161.

[0139] When the damage determination unit 1Dq determines that the damage calculated by the damage calculation processing unit 1Dj is equal to or greater than the damage threshold, the annotation search unit 1Dr searches the classification model 1Ch for an annotation that describes the operating state of the hydraulic excavator 101A at that time. Specifically, the annotation search unit 1Dr searches the classification model 1Ch for an annotation that corresponds to the classification ID associated with the calculated damage. Note that when the damage determination unit 1Dq determines that the damage calculated by the damage calculation processing unit 1Dj is less than the damage threshold, the annotation search unit 1Dr does not search for an annotation.

[0140] The sound output processing unit 1Ds converts the annotation found by the annotation search unit 1Dr into audio data and outputs the audio data to the speaker 126. The speaker 126 outputs the annotation by audio based on the input audio data. The annotation is text data. Therefore, by using software such as TTS (Text to Speech), the processing unit 1Db can output the annotation by audio using the speaker 126 as an audio output device.

[0141] As shown in FIG. 17, the display processing unit 1Do displays a time series graph 15b, which represents the time-series changes in damage to the bucket cylinder, on the display screen of the display device 125. The horizontal axis of the time series graph 15b is the time axis, and displays past history. When the damage determination unit 1Dq determines that damage equal to or greater than the damage threshold has occurred, the display processing unit 1Do displays a time marker 15g on the time axis of the time series graph 15b, indicating the time when damage equal to or greater than the damage threshold occurred. The display processing unit 1Do also displays the time when damage equal to or greater than the damage threshold occurred in the time display area 15e. Therefore, the operator can confirm that significant damage occurred at the time indicated by the time marker 15g and the time displayed in the time display area 15e. The time marker 15g can be moved to specify any past time. The time when damage occurred may be associated with damage information and stored in a database. This allows it to be arbitrarily searched and displayed as damage history information when necessary. The time display area 15e may also display the current time.

[0142] When the damage determination unit 1Dq determines that damage equal to or greater than the damage threshold has occurred, the display processing unit 1Do displays the text "Serious damage has occurred to bucket cylinder" in the damage determination result display area 15c on the display screen of the display device 125. Furthermore, when the damage determination unit 1Dq determines that damage equal to or greater than the damage threshold has occurred, the sound output processing unit 1Ds causes the speaker 126 to output the message "Serious damage has occurred to bucket cylinder" as sound.

[0143] The display processing unit 1Do displays the annotation found based on the feature vector at the time when the damage determination unit 1Dq determines that damage equal to or greater than the damage threshold has occurred in the annotation display area 15d on the display screen of the display device 125. For example, the display processing unit 1Do displays the annotation "Bucket cylinder MAX, jack up" on the display screen of the display device 125. The sound output processing unit 1Ds outputs the annotation "Bucket cylinder MAX, jack up" as sound from the speaker 126. Jacking up refers to the act of lifting the vehicle body using the working device 4.

[0144] The words and phrases that make up the annotation may be changed by an operator or administrator. In other words, the branching factors may be verbalized by combining words and phrases set by the operator or administrator. The collection device 1a or the processing device 1b generates words and phrases based on input information from the input devices 120, 161 (see FIG. 1) and stores them in the classification model 1Ch. This configuration makes it possible to add new words and phrases corresponding to new operating states to the classification model 1Ch or to change words and phrases that are already stored. As a result, it is possible to output annotations that make it easier to understand the operating state of the hydraulic excavator 101A.

[0145] It is preferable that an operator or administrator inputs a phrase as follows. For example, the motion state calculation device 1 displays feature vectors representing two branched motion states on the display devices 125, 165. This allows a person viewing the display screen of the display devices 125, 165 to input a phrase inferred from the difference in feature quantities constituting the two feature vectors using the input devices 120, 161. The motion state calculation device 1 also displays 3D-CG (Computer Graphics) animations representing the two motion states on the display devices 125, 165. This allows a person viewing the display screen of the display devices 125, 165 to input a phrase inferred from the difference between the two animations using the input devices 120, 161. An example of a motion state that can be easily understood using animation is jacking up.

[0146] For example, if the first phrase "bucket cylinder MAX" in "bucket cylinder MAX, jack up" is changed to "at bucket cylinder MAX" and the second phrase "jack up" is changed to "jack up performed," the annotation becomes "jack up performed with bucket cylinder MAX." Also, if the first phrase "bucket cylinder MAX" in "bucket cylinder MAX, jack up" is changed to "with bucket cylinder fully extended" and the second phrase "jack up" is changed to "jack up operation performed," the annotation becomes "jack up operation performed with bucket cylinder fully extended."

[0147] The annotation for display and the annotation for audio output may be different. This makes it possible to shorten the annotation for display and lengthen the annotation for audio output when, for example, the size of the display screen is limited. For example, it is possible to display "Bucket cylinder MAX, jack up" in the annotation display area 15d of the display device 125 and output an audio message from the speaker 126 saying, "The jack-up operation was performed with the bucket cylinder fully extended."

[0148] The display processing unit 1Do uses CG (Computer Graphics) or the like to display an animation of the hydraulic excavator 101A in an animation display area 15f on the display screen of the display device 125. The display processing unit 1Do reproduces the posture of the hydraulic excavator 101A based on the joint angles (boom angle, arm angle, bucket angle) of the working implement 4 and the tilt angle of the vehicle body acquired by the collection device 1Da. The display processing unit 1Do may superimpose an image 15h that highlights areas where significant damage has occurred on the animation. Although not shown, the display processing unit 1Do may also display a tree diagram (see FIGS. 10, 12, and 13) on the display screen of the display device 125.

[0149] As described above, the operational state calculation device 1 according to the fourth embodiment is equipped with a speaker (audio output device) 126 that outputs audio. The processing device 1Db outputs an annotation by audio from the speaker 126. The processing device 1Db determines whether or not the damage suffered by the work machine 101 is equal to or greater than the damage threshold. If the processing device 1Db determines that the damage suffered by the work machine 101 is equal to or greater than the damage threshold, it searches the classification model 1Ch for an annotation that represents the operational state of the work machine 101 at that time, converts the found annotation into audio data, and outputs the annotation by audio from the speaker 126. According to this configuration, when major damage occurs, it is possible to convert an annotation that represents the operational state of the hydraulic excavator 101A into audio and notify the operator of the hydraulic excavator 101A. As a result, when major damage occurs, the operator of the hydraulic excavator 101A can immediately know the operational state of the hydraulic excavator 101A at that time.

[0150] Depending on the location of damage and the operating state, it may be difficult to properly express the location of damage and the operating state of the hydraulic excavator 101A in animation. In this embodiment, the location of damage and the operating state can be output as text or audio, so that the operator can properly know the location of damage and the operating state.

[0151] <Modification of the Fourth Embodiment> In the fourth embodiment, a case has been described in which the work machine 101 is a hydraulic excavator 101A, but the present invention is not limited to this. In this modified example, a case in which the work machine 101 is a dump truck 101B will be described. Fig. 18 is a diagram showing an example of information output from a display device 125 and a speaker 126 provided in the cab of the dump truck 101B. As shown in Fig. 18, a display device 125 and a speaker 126 are provided in the cab of the dump truck 101B.

[0152] The display processing unit 1Do displays a time series graph 15b showing the time series changes in damage to the vehicle body on the display screen of the display device 125. The damage to the vehicle body is calculated, for example, based on changes in pressure of the suspension connecting the tires and the vehicle body. The suspension pressure is detected by a pressure sensor and collected by the second collection unit 1Dg. The damage calculation processing unit 1Dj calculates the damage to the vehicle body based on the changes in suspension pressure collected by the second collection unit 1Dg.

[0153] When the work machine 101 is a dump truck 101B, the feature amounts that make up the feature vector are set to include "vehicle speed," "load weight," "suspension pressure," "hoist cylinder pressure," and the like, which are detected by the detection device.

[0154] Therefore, similar to the above embodiment, the processing device 1Db of this modified example can calculate a feature vector using the operational information of the dump truck 101B as a feature quantity, and assign a classification ID to the calculated feature vector based on the classification model 1Ch.

[0155] In the illustrated example, a classification ID is assigned to the feature vector when a dump truck 101B loaded with transported goods passes over a road surface with large irregularities at high speed, a annotation corresponding to that classification ID is searched for in the classification model 1Ch, and the found annotation is output as voice and displayed as text. In this way, the operating state calculation device 1 can perform the same processing as in the above embodiment even when the work machine 101 is a dump truck 101B.

[0156] The following modified examples are also within the scope of the present invention, and it is possible to combine the configuration shown in the modified example with the configuration described in the above embodiment, to combine the configurations described in the different embodiments above, or to combine the configurations described in the different modified examples below.

[0157] <Variation 1> The processing device 1b according to the first embodiment has been described as displaying a tree diagram (see FIGS. 12 and 13) on the display device 165 and hierarchically displaying the words and phrases that make up the annotation sentences on the tree diagram. The processing device 1b may also display the words and phrases that make up the annotation sentences hierarchically by cataloging them. Here, cataloging refers to listing the operating states hierarchically, like a table of contents. FIG. 19 is a diagram showing an example of a display image of annotation sentences and damage amounts hierarchically displayed by cataloging.

[0158] In the example shown in Fig. 19, the phrase "sideways" corresponding to the magnitude of feature Ca identified as a branching factor in the upper group is displayed as the first item. The phrases "right" and "left" corresponding to the magnitude of feature Cb identified as a branching factor in the middle group are displayed as the second item. The phrases "arm angle vertical" and "arm angle MAX" corresponding to the magnitude of feature Cc identified as a branching factor in the lower group are displayed as the third item.

[0159] The display processing unit 1o displays the average, maximum, and cumulative damage amounts for sections to which the same classification ID is set on the display device 165. In this manner, in this modified example, annotations that indicate the action states are displayed in a catalog, and the damage amount is also displayed. This allows a person viewing the display device 165 (such as an administrator) to easily understand that even if the same "side hit" is performed, a certain action state will cause significant damage.

[0160] In the illustrated example, the "side impact" operation of impacting the side of the working implement 4 of the hydraulic excavator 101A against the side of the natural ground or the like is hierarchically classified and cataloged, but other operations may also be classified and cataloged. For example, jacking up, which uses the working implement 4 to lift the vehicle body, may be hierarchically classified, cataloged, and displayed.

[0161] <Variation 2> In the first to third embodiments, an example was described in which the collection device (machine control device) 1a is provided on the work machine 101, and the processing devices (management servers) 1b, 1Bb, 1Cb are provided on the management facility. In the fourth embodiment, an example was described in which the collection device (machine control device) 1Da and processing device (machine control device) 1Db are provided on the work machine 101. However, the present invention is not limited to this. The management server 150 may have some or all of the functions of the collection device 1a. Also, the machine control device 110 may have some or all of the functions of the processing device 1b. In other words, the work machine operating state calculation device (operating state calculation system) 1 may be provided only on the management facility, or only on the work machine 101. Also, part of the work machine operating state calculation device (operating state calculation system) 1 may be provided on the work machine 101, and the rest may be provided on the management facility.

[0162] For example, the operating state calculation device 1 can also be configured as shown in Fig. 20. In the example shown in Fig. 20, a machine control device 110 provided on a work machine 101 has the functions of the first collection unit 1f, second collection unit 1g, classification processing unit 1i, damage calculation processing unit 1j, and linking processing unit 1k described in the first embodiment, as well as the functions of the damage determination unit 1Dq, annotation text search unit 1Dr, sound output processing unit 1Ds, and display processing unit 1Do described in the fourth embodiment. Furthermore, a management server 150 provided in a management facility has the functions of the statistical processing unit 1m and display processing unit 1o described in the first embodiment, the annotation text generation unit 1Bn described in the second embodiment, and the update processing unit 1Cp described in the third embodiment.

[0163] When damage equal to or greater than the damage threshold occurs, the motion state calculation device 1 according to this modification, like the fourth embodiment, displays an image of the annotation on the display device 125 in real time (sequential processing) and outputs the annotation by voice from the speaker 126. Furthermore, the motion state calculation device 1 according to this modification can execute a hierarchical clustering process and display the generated annotation on the display devices 125, 165 in a display format with a hierarchical structure like a tree diagram.

[0164] The management server 150, which serves as a processing device according to this modification, calculates the inspection timing of damaged parts based on the representative value of damage calculated for each classification ID, which is the result of statistical processing, and displays the calculation result on the display device 165. This allows a person looking at the display device 165 to know the inspection timing of the parts, thereby enabling an appropriate maintenance plan to be made.

[0165] Furthermore, the management server 150 according to this modification calculates the lifespan of the parts based on the results of statistical processing, and displays the calculation results on the display device 165. This allows a person looking at the display device 165 to know the lifespan of the parts, thereby enabling an appropriate maintenance plan to be made.

[0166] Furthermore, the management server 150 according to this modification calculates data for examining the design conditions of the next model to be developed based on the results of the statistical processing, and displays the calculation results on the display device 165. This allows a person viewing the display device 165 to appropriately examine the design conditions of the next model to be developed.

[0167] The management server 150 displays the calculation results (prediction results) of the inspection timing and lifespan of the parts on the display device 165, and also displays an image in which the annotations are organized into a tree diagram or catalog on the display device 165. By displaying the annotations along with the inspection timing and lifespan of the parts on the display device 165, anyone looking at the display device 165 can make appropriate work plans, development plans, etc. for the work machine 101 based on the relationship between the degree of damage, inspection timing, lifespan, and operating status.

[0168] The management server 150 executes update processing for the classification model (master) based on feature vectors acquired from multiple work machines 101. The management server 150 updates the classification model of the work machine 101 based on the classification model (master) periodically, or when requested by the work machine 101.

[0169] According to this configuration, a feature vector that is not similar to the representative feature vector of the classification model is calculated on one of the multiple work machines 101 and transmitted to the management server 150, whereby the new representative feature vector is registered in the classification model (master), and the registered new representative feature vector is also added to the classification model of each work machine 101.

[0170] <Variation 3> In the first to fourth embodiments, the case where the work machine 101 is a hydraulic excavator 101A has been described, and in the modified example of the fourth embodiment, the case where the work machine 101 is a dump truck 101B has been described, but the present invention is not limited to this. The present invention can be applied to various work machines equipped with work implements that perform work such as excavation, transport, assembly, and disassembly.

[0171] <Variation 4> In the above embodiment, an example has been described in which classification IDs are reclassified into multiple groups in a hierarchical structure by hierarchical clustering processing, but the present invention is not limited to this. Classification IDs may also be reclassified into multiple groups by a neural network method.

[0172] <Variation 5> In the first embodiment, the processing device 1b extracts category IDs for which the amount of damage is equal to or greater than a representative threshold, and generates annotations for the extracted category IDs. However, the present invention is not limited to this. The processing device 1b may generate annotations for all category IDs regardless of the amount of damage.

[0173] <Variation 6> In the first embodiment, an example in which the annotation is displayed on the display device 165 has been described, but the manner in which the annotation is output to the outside is not limited to this. The motion state calculation device 1 may include a printing device as an output device. In this case, the processing device 1b outputs the annotation to the outside by printing the annotation on paper using the printing device.

[0174] In the first embodiment, an audio output device such as a speaker may be provided in the management facility, and the processing device 1b may convert the annotation into audio data and output the converted audio data to the audio output device, thereby enabling the annotation to be output as audio by the audio output device.

[0175] Furthermore, the processing device 1b may transmit (output) the action information including the annotation to a mobile terminal such as a smartphone, a tablet PC, or a mobile phone via the communication device 162 as an output device. This allows the owner of the mobile terminal to display the information including the annotation on the display screen by operating the mobile terminal.

[0176] <Variation 7> In the first embodiment, an example has been described in which the processing device 1b generates an annotation sentence by connecting a phrase corresponding to the magnitude of the identified feature amount Ca, a phrase corresponding to the magnitude of the identified feature amount Cb, and a phrase corresponding to the magnitude of the identified feature amount Cc, i.e., an example in which an annotation sentence is generated by connecting three phrases. However, the present invention is not limited to this. The processing device 1b may generate an annotation sentence by connecting at least two phrases. Furthermore, the processing device 1b is not limited to performing hierarchical clustering processing with first to third hierarchies. The processing device 1b may also perform hierarchical clustering processing with four or more hierarchies. In this case, the annotation sentence is generated by connecting four or more phrases.

[0177] <Variation 8> In the first embodiment, an example has been described in which the damage targeted by the damage calculation processing unit 1j is physical damage such as stress fatigue damage, but the present invention is not limited to this. Damage targeted by the damage calculation processing unit 1j may also include situations in which the hydraulic excavator 101A becomes unstable, such as a situation in which the hydraulic excavator 101A is in an unbalanced position. An unbalanced position of the hydraulic excavator 101A can be detected by attaching an inclinometer to the hydraulic excavator 101A.

[0178] <Variation 9> In the first embodiment, an example was described in which the bar graph (see FIG. 11) and the tree diagram (see FIGS. 12 and 13) displayed on the display screen 165a of the display device 165 include a classification ID, but the present invention is not limited to this. The classification ID does not have to be displayed on the bar graph (see FIG. 11) and the tree diagram (see FIGS. 12 and 13). It is sufficient that the display screen 165a of the display device 165 displays at least an image in which an annotation text and a damage amount are associated.

[0179] <Modification 10> In the fourth embodiment, an example was described in which an animation was displayed on the display screen of the display device 125, 165, associating a note with the amount of damage (see FIG. 17 ). However, the display range is not limited to the side (front side) of the work machine 101. Because 3D-CG animation display is an observation video in which the status of the work machine 101 is projected onto the display screen from a certain viewpoint, there is a problem in that if a problematic operation (damaging operation) is occurring on the back side, the situation cannot be grasped. Therefore, a configuration (damage area determination means) may be provided that determines whether the damage cannot be identified in the animation, and the display may be either text or audio depending on the determination result. In other words, the cumulative damage level of the work machine 101 can increase not only on the front side, which can be confirmed in the animation, but also on the back side. However, the overall condition of the work machine can be grasped by displaying the back side, which cannot be confirmed visually. For example, the cumulative damage level of the front side of the work machine 101 may be displayed as an animation, and the back side may be displayed as text or audio. Any combination of display methods may be used. This enables more useful notifications to the operator.

[0180] Although the embodiments of the present invention have been described above, the above embodiments merely illustrate some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above embodiments.

[0181] The above-described embodiments and modifications are provided as examples to facilitate understanding of the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment or modification with the configuration of another embodiment or modification, and it is also possible to add the configuration of another embodiment or modification to the configuration of one embodiment or modification. Note that the control lines and information lines shown in the figures are those considered necessary for explanation, and do not necessarily represent all control lines and information lines required in the product. In reality, it can be assumed that almost all configurations are interconnected. [Explanation of symbols]

[0182] 1...Work machine operation state calculation device, 1a, 1Da...Collection device, 1b, 1Bb, 1Cb, 1Db...Processing device, 1f, 1Df...First collection unit, 1g, 1Dg...Second collection unit, 1h, 1Bh, 1Ch...Classification model, 1i, 1Ci, 1Di...Classification processing unit, 1j, 1Dj...Damage calculation processing unit, 1k, 1Dk...Linking processing unit, 1m...Statistical processing unit, 1n, 1Bn...Annotation sentence generation unit, 1o, 1Do...Display processing unit, 1u...Damage operation database, 1Cp...Update processing unit, 1Dq...Da Image determination unit, 1Dr...annotation text search unit, 1Ds...sound output processing unit, 4...work equipment, 101...work machine, 101A...hydraulic excavator (work machine), 101B...dump truck (work machine), 101C...work robot (work machine), 120...input device, 121 to 124...detection device, 125...display device (output device), 126...speaker (audio output device, output device), 150...management server, 161...input device, 162...communication device (output device), 165...display device (output device)

Claims

1. a processing device that calculates the operating state of the work machine based on a plurality of pieces of operation information associated with the operation of the work machine that changes over time; an output device that outputs operation information that represents the operation state of the work machine calculated by the processing device, The processing device includes: calculating a feature vector having the plurality of pieces of operation information as feature quantities for each of a plurality of time periods; assigning a classification ID to the feature vectors calculated at multiple times based on a classification model in which a representative feature vector, which is a representative value of a cluster of feature vectors that represent the operating state of the work machine, is linked to a classification ID; calculating damage sustained by the work machine at each of a plurality of times based on the plurality of pieces of operation information; storing a database linking a time, the classification ID assigned to the calculated feature vector, and the calculated damage; calculating a representative value of the damage for each classification ID based on the time, the classification ID, and the damage included in the database; Classifying the plurality of classification IDs into a plurality of groups; Identifying a feature to be used for generating a sentence based on a difference between the representative feature vectors associated with the plurality of classification IDs classified into the same group; generating a sentence expressing the operational state of the work machine corresponding to the classification ID by concatenating words and phrases corresponding to the magnitude of the feature amount identified in each of the plurality of groups to which the classification ID belongs; outputting to the output device operation information that associates a sentence that represents the operating state of the work machine with the representative value of the damage calculated for the classification ID linked to the representative feature vector used to generate the sentence; The output device outputs the operation information input from the processing device. A working machine operating state calculation device characterized by:

2. 2. The operating state calculation device for a work machine according to claim 1, The processing device includes: classifying the plurality of classification IDs into a plurality of groups in a hierarchical structure based on the representative feature vector; calculating a representative feature vector of the lower group based on the representative feature vectors linked to a plurality of classification IDs belonging to the same lower group; identifying a first feature amount based on differences between representative feature vectors of a plurality of lower groups that belong to the same upper group; Identifying a second feature amount based on a difference between the representative feature vectors associated with a plurality of classification IDs belonging to the same lower-level group; A sentence is generated that expresses the operational state of the work machine corresponding to the classification ID, and that includes a phrase corresponding to the magnitude of the identified first feature amount and a phrase corresponding to the magnitude of the identified second feature amount. A working machine operating state calculation device characterized by:

3. In the operating state calculation device for a work machine according to claim 1, The processing device includes: classifying the plurality of classification IDs into a plurality of lower groups based on the distances between the representative feature vectors linked to the plurality of classification IDs; calculating a representative feature vector of the lower group based on the representative feature vectors linked to a plurality of classification IDs belonging to the same lower group; classifying the plurality of lower groups into a plurality of upper groups based on distances between the representative feature vectors of the lower groups; comparing a plurality of feature amounts constituting the representative feature vectors of a plurality of lower groups belonging to the same upper group, and identifying the feature amount having the largest difference as a first feature amount; comparing a plurality of feature amounts constituting the representative feature vector linked to a plurality of classification IDs belonging to the same lower group with each other, and identifying the feature amount having the largest difference as a second feature amount; generating a sentence expressing the operational state of the work machine corresponding to the classification ID by concatenating at least a phrase corresponding to the magnitude of the identified first feature amount and a phrase corresponding to the magnitude of the identified second feature amount; Outputting to the output device operation information that associates a sentence that represents the operating state of the work machine corresponding to the classification ID with the representative value of the damage calculated for that classification ID. A working machine operating state calculation device characterized by:

4. 3. The operating state calculation device for a work machine according to claim 2, The output device is a display device that displays an image, The processing device causes the display device to hierarchically display the words and phrases that make up the sentence. A working machine operating state calculation device characterized by:

5. 5. The operating state calculation device for a work machine according to claim 4, The processing device includes: displaying a tree diagram in which the classification IDs are leaves on the display device; The display mode of the branch connected to the leaf in the tree diagram is changed according to the magnitude of the representative value of the damage. A working machine operating state calculation device characterized by:

6. 2. The operating state calculation device for a work machine according to claim 1, The processing device includes: extracting the classification IDs whose representative values of the damage are equal to or greater than a representative threshold value; For the extracted plurality of classification IDs, a sentence expressing the operating state of the work machine corresponding to the classification ID is generated. A working machine operating state calculation device characterized by:

7. 2. The operating state calculation device for a work machine according to claim 1, a storage device that stores the sentences in association with the classification IDs, The processing device includes: determining whether the sentence associated with the classification ID is stored in the storage device; If it is determined that the sentence linked to the category ID is not stored in the storage device, a sentence that represents the operating state of the work machine corresponding to the category ID is generated, the generated sentence is linked to the category ID and stored in the storage device, and operation information that associates the generated sentence with the representative value of the damage calculated for the category ID linked to the generated sentence is output to the output device. When it is determined that the sentence linked to the classification ID is stored in the storage device, the sentence stored in the storage device is selected without generating a sentence that represents the operational state of the work machine corresponding to the classification ID, and operation information that associates the selected sentence with the representative value of the damage calculated for the classification ID linked to the selected sentence is output to the output device. A working machine operating state calculation device characterized by:

8. 2. The operating state calculation device for a work machine according to claim 1, The processing device calculates a representative value of the damage for each work environment. A working machine operating state calculation device characterized by:

9. 2. The operating state calculation device for a work machine according to claim 1, The processing device includes: determining whether a representative feature vector similar to the calculated feature vector is stored in the classification model; If it is determined that a representative feature vector similar to the calculated feature vector is not stored in the classification model, a new representative feature vector and a new classification ID are generated based on the calculated feature vector, and the generated new representative feature vector and the new classification ID are linked and stored in the classification model. A working machine operating state calculation device characterized by:

10. The operating state calculation device for a work machine according to claim 7, The output device is an audio output device that outputs audio, The processing device includes: determining whether or not the damage sustained by the work machine is equal to or greater than a damage threshold; When it is determined that the damage received by the work machine is equal to or greater than a damage threshold, the sentence representing the operating state of the work machine at that time is output by voice from the voice output device. A working machine operating state calculation device characterized by:

11. 2. The operating state calculation device for a work machine according to claim 1, The processing device calculates at least one of an inspection timing and a lifespan of the damaged part based on the representative value of the damage calculated for each of the classification IDs, and outputs the statement together with the calculation result by the output device. A working machine operating state calculation device characterized by:

12. 2. The operating state calculation device for a work machine according to claim 1, The processing device generates and stores the phrase based on input information from an input device. A working machine operating state calculation device characterized by:

Images